Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19041#discussion_r143799104
--- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala
---
@@ -0,0 +1,250 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19041#discussion_r143800553
--- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala
---
@@ -0,0 +1,250 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143819790
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd =
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18966
LGTM pending Jenkins
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19082
I agree with you. #18810 compares the following two code.
1. Interpreter execution of Java code by whole-stage codegen with passing
row data in scalar values
2. JITted execution of Java code
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18966
**[Test build #82598 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82598/testReport)**
for PR 18966 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19309
**[Test build #82597 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82597/testReport)**
for PR 19309 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18732
**[Test build #82599 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82599/testReport)**
for PR 18732 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19309
Let me summarize the PR. Please correct me if anything is missing.
**Background**
Currently, our users can register a listener one by one during the
executions by calling the
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19309
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143813642
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143812619
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala
---
@@ -44,14 +73,18 @@ case class
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143812311
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala ---
@@ -435,6 +435,35 @@ class RelationalGroupedDataset
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19082
You know, when we disable the whole-stage codegen, we still do the
expression codegen, which byte code size is smaller than 8K at most cases.
Thus, it could be faster than whole-stage codegen.
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143810948
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd = self._jgd.pivot(pivot_col)
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143810736
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd = self._jgd.pivot(pivot_col)
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143810539
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd = self._jgd.pivot(pivot_col)
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143810355
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd = self._jgd.pivot(pivot_col)
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143804627
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143803904
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143803469
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143810078
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143808170
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143809031
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143810175
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143805303
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143809711
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd = self._jgd.pivot(pivot_col)
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18966#discussion_r143808338
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeFormatter.scala
---
@@ -89,6 +89,14 @@ object CodeFormatter
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18966
**[Test build #82596 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82596/testReport)**
for PR 18966 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18732
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18732
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82587/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18732
**[Test build #82587 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82587/testReport)**
for PR 18732 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19250
**[Test build #82595 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82595/testReport)**
for PR 19250 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19439
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82591/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19439
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19439
**[Test build #82591 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82591/testReport)**
for PR 19439 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/18732
I had some minor comments on the docs, otherwise LGTM!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143803982
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143802697
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala
---
@@ -44,14 +73,18 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19424
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user eyalfa commented on the issue:
https://github.com/apache/spark/pull/19181
@hvanhovell , thanks :+1:
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143802019
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala ---
@@ -435,6 +435,35 @@ class RelationalGroupedDataset
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19424
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82586/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19181
**[Test build #82594 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82594/testReport)**
for PR 19181 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19424
**[Test build #82586 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82586/testReport)**
for PR 19424 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18732
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82585/
Test PASSed.
---
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/19181
I will merge this when it passes tests.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18732
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18732
**[Test build #82585 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82585/testReport)**
for PR 18732 at commit
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143800589
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd =
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143800072
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd =
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18732
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82584/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18732
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143799780
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd =
Github user MrBago commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143799505
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,229 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user MrBago commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143794449
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,229 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user MrBago commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143799083
--- Diff: python/pyspark/ml/image.py ---
@@ -0,0 +1,133 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+#
Github user MrBago commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143798662
--- Diff: python/pyspark/ml/image.py ---
@@ -0,0 +1,133 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+#
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18732
**[Test build #82584 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82584/testReport)**
for PR 18732 at commit
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18732#discussion_r143799187
--- Diff: python/pyspark/sql/group.py ---
@@ -192,7 +193,84 @@ def pivot(self, pivot_col, values=None):
jgd =
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/18966#discussion_r143798289
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -769,16 +769,21 @@ class CodegenContext {
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19082
The whole-stage codegen has two advantages according to [this
paper](http://www.vldb.org/pvldb/vol4/p539-neumann.pdf).
1. enable compiler optimizations among operations (3. in page 2)
2. pass
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19269
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82589/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19269
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19269
**[Test build #82589 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82589/testReport)**
for PR 19269 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19269
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82588/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19269
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19269
**[Test build #82588 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82588/testReport)**
for PR 19269 at commit
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/19424
What are the guarantees made by the previous batches in the optimizer? The
work done by `FilterAndProject` seems redundant to me because the optimizer
should already push filters below projection.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19438
**[Test build #82593 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82593/testReport)**
for PR 19438 at commit
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19459#discussion_r143788090
--- Diff: python/pyspark/sql/session.py ---
@@ -510,9 +511,43 @@ def createDataFrame(self, data, schema=None,
samplingRatio=None, verifySchema=Tr
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/19269
> There is no restriction to let the output of data writers be visible to
other writers, so it's possible to launch a write task just for cleaning up the
data of other writers.
Agreed.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19250
**[Test build #82592 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82592/testReport)**
for PR 19250 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19439
**[Test build #82591 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82591/testReport)**
for PR 19439 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19082
@kiszk Thanks for your summary. I had a few related discussions with
@rednaxelafx and @liancheng in the recent weeks. Vertical cuts like
https://github.com/apache/spark/pull/19082 is pretty
Github user imatiach-msft commented on the issue:
https://github.com/apache/spark/pull/19439
@viirya thank you for the great comments, I've updated the PR. I'm waiting
to hear back from @dakirsa on the source of the two BGR and BGRA images.
---
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143783349
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143782986
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143782832
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143782452
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143782366
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143781910
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19222
ping @hvanhovell @tejasapatil
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143779796
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19451
**[Test build #82590 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82590/testReport)**
for PR 19451 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19451
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82590/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19451
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143778773
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19451
**[Test build #82590 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82590/testReport)**
for PR 19451 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19451
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19451#discussion_r143776877
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1242,6 +1243,53 @@ object
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143773560
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143773163
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19181#discussion_r143773051
--- Diff:
core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java
---
@@ -503,6 +504,41 @@ public void
Github user imatiach-msft commented on a diff in the pull request:
https://github.com/apache/spark/pull/19439#discussion_r143772255
--- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software
Github user imatiach-msft commented on the issue:
https://github.com/apache/spark/pull/19439
"I saw there are few images, just want to make sure, are those images are
safe of license issue to be included in Spark?"
In the original spark package we used images from CIFAR-10, but we
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19181#discussion_r143771647
--- Diff:
core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorterSuite.java
---
@@ -139,4 +139,49 @@ public int compare(
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19181#discussion_r143771458
--- Diff:
core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorterSuite.java
---
@@ -139,4 +139,49 @@ public int compare(
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19181#discussion_r143770712
--- Diff:
core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java
---
@@ -503,6 +504,41 @@ public void
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/19181#discussion_r143770192
--- Diff:
core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java
---
@@ -503,6 +504,41 @@ public void
301 - 400 of 528 matches
Mail list logo
