Github user angolon commented on the issue:
https://github.com/apache/spark/pull/14710
Hrmm... SparkContextSuite passes all tests for me locally. Any idea what
might be happening here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14698
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64395/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14698
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14698
**[Test build #64395 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64395/consoleFull)**
for PR 14698 at commit
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14702#discussion_r76183051
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/script/ScriptTransformationExec.scala
---
@@ -0,0 +1,313 @@
+/*
+ * Licensed to the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/8880
**[Test build #64401 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64401/consoleFull)**
for PR 8880 at commit
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14702
This looks reasonable.
cc @hvanhovell to take a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14702#discussion_r76183006
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/script/ScriptTransformationExec.scala
---
@@ -0,0 +1,313 @@
+/*
+ * Licensed to the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14800
**[Test build #64400 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64400/consoleFull)**
for PR 14800 at commit
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14753#discussion_r76181435
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/TypedImperativeAggregateSuite.scala
---
@@ -0,0 +1,300 @@
+/*
+ * Licensed to the Apache
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14537
**[Test build #64399 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64399/consoleFull)**
for PR 14537 at commit
Github user mpjlu commented on the issue:
https://github.com/apache/spark/pull/14785
Sure, I will fix it, and add test cases. thanks. @dbtsai ,
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14710
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64394/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14710
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14710
**[Test build #64394 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64394/consoleFull)**
for PR 14710 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14753
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14753
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64393/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14744
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14744
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64391/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14753
**[Test build #64393 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64393/consoleFull)**
for PR 14753 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14744
**[Test build #64391 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64391/consoleFull)**
for PR 14744 at commit
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/14785
Can you also fix
`https://github.com/apache/spark/blob/master/mllib-local/src/test/scala/org/apache/spark/ml/util/TestingUtils.scala`?
Please add tests showing the issue is addressed. Thanks.
---
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14537#discussion_r76180471
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala ---
@@ -372,6 +373,40 @@ class OrcQuerySuite extends QueryTest with
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14537#discussion_r76180414
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala ---
@@ -54,10 +57,12 @@ class OrcFileFormat extends FileFormat with
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/9
**[Test build #64398 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64398/consoleFull)**
for PR 9 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14537
**[Test build #64397 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64397/consoleFull)**
for PR 14537 at commit
Github user rajeshbalamohan commented on the issue:
https://github.com/apache/spark/pull/14537
Thanks @gatorsmile, it would be good to retain the change in
OrcFileInputFormat's inferschema (just in case it is referenced later).
---
If your project is set up for it, you can reply to
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14757
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14800
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/14757
thanks. merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14800
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64392/
Test PASSed.
---
Github user rajeshbalamohan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14537#discussion_r76179877
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala ---
@@ -54,10 +57,12 @@ class OrcFileFormat extends FileFormat
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14800
**[Test build #64392 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64392/consoleFull)**
for PR 14800 at commit
Github user HyukjinKwon closed the pull request at:
https://github.com/apache/spark/pull/14799
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14799
Sure! thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14789
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14789
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64390/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14789
**[Test build #64390 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64390/consoleFull)**
for PR 14789 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/14800
`SampleExec` possibly generates multiple rows from a single row when
`withReplacement=true`.
So, `CodegenContext#copyResult` should be `true`.
See:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/14800
@HyukjinKwon thanks your comment. I'll add the requirement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14746
**[Test build #64396 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64396/consoleFull)**
for PR 14746 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14795
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14800
Thank you for cc me @maropu
I like this change. BTW, we may have to add a condition for enforcing
sampling ratio <= 1.0 too.
---
If your project is set up for it, you can reply to this
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14800
Can you describe the bug?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14573
**[Test build #3233 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3233/consoleFull)**
for PR 14573 at commit
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14799
Thanks - merging in. Can you close this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14573
cc @hvanhovell for another look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14573#discussion_r76177989
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -116,6 +116,13 @@ object SQLConf {
.longConf
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14753#discussion_r76177616
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/TypedImperativeAggregateSuite.scala
---
@@ -0,0 +1,300 @@
+/*
+ * Licensed to the Apache
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14746#discussion_r76177293
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala
---
@@ -204,10 +206,67 @@ class SQLViewSuite extends QueryTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14746#discussion_r76177170
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala
---
@@ -204,10 +206,67 @@ class SQLViewSuite extends QueryTest
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14746#discussion_r76177107
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala
---
@@ -204,10 +206,67 @@ class SQLViewSuite extends QueryTest
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14698#discussion_r76177037
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala
---
@@ -136,7 +136,7 @@ trait
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14746#discussion_r76177017
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala
---
@@ -204,10 +206,67 @@ class SQLViewSuite extends QueryTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14746#discussion_r76176938
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala
---
@@ -204,10 +206,67 @@ class SQLViewSuite extends QueryTest
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/14796#discussion_r76176921
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala ---
@@ -66,6 +70,7 @@ private object PostgresDialect extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14746#discussion_r76176843
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ---
@@ -105,7 +105,13 @@ case class CreateViewCommand(
}
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14799
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14799
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64389/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14799
**[Test build #64389 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64389/consoleFull)**
for PR 14799 at commit
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14698#discussion_r76176582
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
---
@@ -474,6 +474,20 @@ case class MapObjects
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14698#discussion_r76176311
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
---
@@ -474,6 +474,20 @@ case class MapObjects
Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14698#discussion_r76176321
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
---
@@ -474,6 +474,20 @@ case class MapObjects
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14753#discussion_r76176167
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/TypedImperativeAggregateSuite.scala
---
@@ -0,0 +1,300 @@
+/*
+ * Licensed to the Apache
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14698
**[Test build #64395 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64395/consoleFull)**
for PR 14698 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/10896
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64388/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/10896
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/10896
**[Test build #64388 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64388/consoleFull)**
for PR 10896 at commit
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14753#discussion_r76175290
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggregationIterator.scala
---
@@ -234,7 +234,24 @@ abstract class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14710
**[Test build #64394 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64394/consoleFull)**
for PR 14710 at commit
Github user winningsix commented on a diff in the pull request:
https://github.com/apache/spark/pull/8880#discussion_r76175174
--- Diff:
yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnIOEncryptionSuite.scala ---
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/14744#discussion_r76174242
--- Diff: docs/configuration.md ---
@@ -1752,6 +1752,14 @@ showDF(properties, numRows = 200, truncate = FALSE)
Executable for executing R scripts in
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14753
**[Test build #64393 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64393/consoleFull)**
for PR 14753 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14753#discussion_r76173845
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggregationIterator.scala
---
@@ -234,7 +234,24 @@ abstract class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14800
**[Test build #64392 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64392/consoleFull)**
for PR 14800 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/14800
@HyukjinKwon @rxin could you check this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/14800
[SPARK-15382][SQL] Fix a bug in sampling with replacement
## What changes were proposed in this pull request?
This pr to fix a bug below in sampling with replacement
```
val df = Seq((1,
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14744#discussion_r76173164
--- Diff: docs/configuration.md ---
@@ -1752,6 +1752,14 @@ showDF(properties, numRows = 200, truncate = FALSE)
Executable for executing R
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14744
**[Test build #64391 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64391/consoleFull)**
for PR 14744 at commit
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/14744#discussion_r76172717
--- Diff: docs/configuration.md ---
@@ -1752,6 +1752,14 @@ showDF(properties, numRows = 200, truncate = FALSE)
Executable for executing R scripts in
Github user zjffdu commented on a diff in the pull request:
https://github.com/apache/spark/pull/14744#discussion_r76172436
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java
---
@@ -336,7 +336,8 @@
join(File.separator,
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14784#discussion_r76171608
--- Diff: R/pkg/R/sparkR.R ---
@@ -365,6 +365,10 @@ sparkR.session <- function(
}
overrideEnvs(sparkConfigMap, paramMap)
}
+
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14176
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64387/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14176
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14176
**[Test build #64387 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64387/consoleFull)**
for PR 14176 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14789
**[Test build #64390 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64390/consoleFull)**
for PR 14789 at commit
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14784#discussion_r76171359
--- Diff: R/pkg/R/sparkR.R ---
@@ -503,6 +507,10 @@ sparkConfToSubmitOps[["spark.driver.memory"]]
<- "--driver-memory"
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14784#discussion_r76171254
--- Diff: R/pkg/R/sparkR.R ---
@@ -365,6 +365,10 @@ sparkR.session <- function(
}
overrideEnvs(sparkConfigMap, paramMap)
}
+
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14784#discussion_r76171185
--- Diff: R/pkg/R/sparkR.R ---
@@ -503,6 +507,10 @@ sparkConfToSubmitOps[["spark.driver.memory"]]
<- "--driver-memory"
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14744#discussion_r76171033
--- Diff: docs/configuration.md ---
@@ -1752,6 +1752,14 @@ showDF(properties, numRows = 200, truncate = FALSE)
Executable for executing R
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14744#discussion_r76170947
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java
---
@@ -336,7 +336,8 @@
join(File.separator,
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13152
Thanks - this looks pretty good!
I've triggered a new Jenkins run and also left some small comments. It
would be great to add some unit tests (not integration tests) for two of the
classes
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13152#discussion_r76170909
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockReplicationPolicy.scala ---
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13152#discussion_r76170841
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockReplicationPolicy.scala ---
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13152#discussion_r76170799
--- Diff: core/src/main/scala/org/apache/spark/storage/TopologyMapper.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13152#discussion_r76170783
--- Diff:
core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
---
@@ -37,6 +35,8 @@ import
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13152#discussion_r76170745
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala
---
@@ -55,10 +55,22 @@ class BlockManagerMasterEndpoint(
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14613
@sun-rui @shivaram thought?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/13152#discussion_r76170720
--- Diff: core/src/main/scala/org/apache/spark/storage/TopologyMapper.scala
---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14613#discussion_r76170702
--- Diff: R/pkg/R/types.R ---
@@ -67,3 +67,27 @@ rToSQLTypes <- as.environment(list(
"double" = "double",
"character" = "string",
1 - 100 of 600 matches
Mail list logo
