Github user saeedehA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-167496362
Hello
I'm sorry that I'm beginner in pyspark, but need to get .bin file as byte
array and I don't understand anything about what I must do, is it possible that
say
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r22288031
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -510,6 +510,52 @@ class SparkContext(config: SparkConf) extends Logging {
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/1658
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61379142
Thanks @kmader, I merged this now. I manually amended the patch a bit to
fix style issues (there were still a bunch of commas without spaces, etc), and
I also changed the
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-60979360
@kmader btw if you don't have time to deal with these comments, let me
know; I might be able to take the patch from where it is and implement them.
---
If your project
Github user kmader commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r19582168
--- Diff: core/src/main/scala/org/apache/spark/rdd/BinaryFileRDD.scala ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61030284
[Test build #22503 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22503/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61031018
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61031930
Thanks for the update, Kevin. Looks like Jenkins had some issues with git,
will retry it.
---
If your project is set up for it, you can reply to this email and have your
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61031939
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61032240
[Test build #22505 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22505/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61035841
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61035835
[Test build #22503 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22503/consoleFull)
for PR 1658 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61037379
[Test build #22505 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22505/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-61037383
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r19453138
--- Diff:
core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala ---
@@ -220,6 +227,83 @@ class JavaSparkContext(val sc: SparkContext) extends
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-60709302
Thanks for the update, Kevin. Note that there are still a few comments from
me on https://github.com/apache/spark/pull/1658/files, do you mind dealing with
those?
---
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r19453170
--- Diff: core/src/main/scala/org/apache/spark/rdd/BinaryFileRDD.scala ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59882709
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21972/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59887629
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59887621
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21972/consoleFull)
for PR 1658 at commit
Github user kmader commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r19133684
--- Diff:
core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala ---
@@ -220,6 +227,83 @@ class JavaSparkContext(val sc: SparkContext) extends
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-60015558
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22011/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-60015564
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59805377
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21922/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59805569
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59805565
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21922/consoleFull)
for PR 1658 at commit
Github user kmader commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59832070
So I made the requested changes and added a few more tests, but the tests
appear to have not run for a strange reason:
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59866772
There might've been some Jenkins issues recently; going to restart it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59866801
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59866791
BTW for the style, you can do sbt/sbt scalastyle locally if you want. Not
sure there's a command in Maven.
---
If your project is set up for it, you can reply to this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59867124
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21950/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-59867194
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18610904
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -511,6 +511,67 @@ class SparkContext(config: SparkConf) extends Logging {
}
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18610930
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -510,6 +510,53 @@ class SparkContext(config: SparkConf) extends Logging {
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18610978
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,219 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611084
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,219 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611129
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,219 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611204
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,219 @@
+/*
--- End diff --
This file contains several
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611224
--- Diff: core/src/main/scala/org/apache/spark/rdd/BinaryFileRDD.scala ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611326
--- Diff: core/src/main/scala/org/apache/spark/rdd/BinaryFileRDD.scala ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611451
--- Diff: core/src/test/java/org/apache/spark/JavaAPISuite.java ---
@@ -836,6 +839,84 @@ public void sequenceFile() {
Assert.assertEquals(pairs,
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611774
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -224,6 +226,128 @@ class FileSuite extends FunSuite with
LocalSparkContext {
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18611791
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -224,6 +226,128 @@ class FileSuite extends FunSuite with
LocalSparkContext {
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18612223
--- Diff:
core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala ---
@@ -220,6 +227,83 @@ class JavaSparkContext(val sc: SparkContext) extends
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18612265
--- Diff:
core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala ---
@@ -220,6 +227,83 @@ class JavaSparkContext(val sc: SparkContext) extends
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18612296
--- Diff:
core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala ---
@@ -220,6 +227,83 @@ class JavaSparkContext(val sc: SparkContext) extends
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18612346
--- Diff:
core/src/main/scala/org/apache/spark/input/FixedLengthBinaryRecordReader.scala
---
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18612406
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -224,6 +226,128 @@ class FileSuite extends FunSuite with
LocalSparkContext {
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-58429329
Hey Kevin, sorry for the delay in getting back to this. I just made a few
more comments. I think this is getting pretty close, hopefully we can put it in
1.2.
---
If
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18613097
--- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala ---
@@ -224,6 +226,128 @@ class FileSuite extends FunSuite with
LocalSparkContext {
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18613144
--- Diff: core/src/test/java/org/apache/spark/JavaAPISuite.java ---
@@ -836,6 +839,84 @@ public void sequenceFile() {
Assert.assertEquals(pairs,
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18613166
--- Diff:
core/src/main/scala/org/apache/spark/input/FixedLengthBinaryRecordReader.scala
---
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18613254
--- Diff: core/src/test/java/org/apache/spark/JavaAPISuite.java ---
@@ -836,6 +839,84 @@ public void sequenceFile() {
Assert.assertEquals(pairs,
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18613290
--- Diff: core/src/main/scala/org/apache/spark/rdd/BinaryFileRDD.scala ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user kmader commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18335807
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57623981
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21200/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57632461
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57632451
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21200/consoleFull)
for PR 1658 at commit
Github user kmader commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18267344
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user kmader commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18267674
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -511,6 +511,67 @@ class SparkContext(config: SparkConf) extends Logging {
}
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57440499
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21103/consoleFull)
for PR 1658 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57440605
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21103/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57440612
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57451347
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21108/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57451427
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57451425
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21108/consoleFull)
for PR 1658 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57452596
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21109/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57457522
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57457518
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21109/consoleFull)
for PR 1658 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57458416
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21110/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57465007
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57464997
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21110/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57469082
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57475609
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21117/consoleFull)
for PR 1658 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57489219
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21117/consoleFull)
for PR 1658 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-57489237
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18300208
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r18315773
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707338
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707347
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707360
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707399
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707460
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -511,6 +511,67 @@ class SparkContext(config: SparkConf) extends Logging {
}
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-55990853
Jenkins, add to whitelist and test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707495
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-55990912
@kmader sorry for the delay on this, but the new version looks pretty good.
Made a few more comments on the API. The other thing I see is that the code
style is not quite
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707506
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707520
--- Diff: core/src/main/scala/org/apache/spark/input/RawFileInput.scala ---
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-55990964
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20521/consoleFull)
for PR 1658 at commit
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707533
--- Diff: core/src/test/java/org/apache/spark/JavaAPISuite.java ---
@@ -836,6 +838,29 @@ public void sequenceFile() {
Assert.assertEquals(pairs,
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17707574
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -511,6 +511,67 @@ class SparkContext(config: SparkConf) extends Logging {
}
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-55993418
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20521/consoleFull)
for PR 1658 at commit
Github user freeman-lab commented on a diff in the pull request:
https://github.com/apache/spark/pull/1658#discussion_r17709043
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -511,6 +511,67 @@ class SparkContext(config: SparkConf) extends Logging {
Github user jrabary commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-55714177
Hi all,
I'm trying to use this patch to load a set of jpeg images but the path
(key) of the output is empty
val image = sc.binaryFiles(data/*.jpg)
Github user kmader commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-55769371
Thanks @jrabary for this find, it had to do with the new method for
handling PortableDataStreams which didn't calculate the name correctly. I
think I have it fixed now
Github user kmader commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-54744540
Hey @mateiz, Sorry, I had other projects to work on. I have made the
changes and called the new class ```PortableDataStream```
---
If your project is set up for it,
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-54694543
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-54699324
Hey @kmader, just curious, did you think about this BinaryData object idea?
If you don't have time to do it, someone else can also pick up from where you
left off.
---
Github user freeman-lab commented on the pull request:
https://github.com/apache/spark/pull/1658#issuecomment-52149331
@kmader great to incorporate the FixedRecord stuff into the PR, thanks! I
like @mateiz's suggestion for naming the two methods. I was starting to work on
1 - 100 of 122 matches
Mail list logo
