Github user MaxGekk closed the pull request at:
https://github.com/apache/spark/pull/21589
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r203235747
--- Diff: R/pkg/R/context.R ---
@@ -437,3 +437,33 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r203235671
--- Diff: R/pkg/R/context.R ---
@@ -437,3 +437,33 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202545679
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala ---
@@ -67,6 +67,10 @@ private[spark] trait TaskScheduler {
// Get
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202503533
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends
Logging {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202503503
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202462678
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202460774
--- Diff: python/pyspark/context.py ---
@@ -406,6 +406,22 @@ def defaultMinPartitions(self):
"""
return
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202459283
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends
Logging {
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r202454060
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends
Logging {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201973111
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201968413
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201920416
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201914125
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201913817
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201913578
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201887843
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends
Logging {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201887513
--- Diff:
core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala ---
@@ -128,6 +128,18 @@ class JavaSparkContext(val sc: SparkContext)
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201887443
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2336,6 +2336,18 @@ class SparkContext(config: SparkConf) extends
Logging {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201887249
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201887142
--- Diff: python/pyspark/context.py ---
@@ -406,6 +406,22 @@ def defaultMinPartitions(self):
"""
return
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201887215
--- Diff: python/pyspark/context.py ---
@@ -406,6 +406,22 @@ def defaultMinPartitions(self):
"""
return
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198800988
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198723017
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198722933
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198722972
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198708461
--- Diff: python/pyspark/context.py ---
@@ -406,6 +406,22 @@ def defaultMinPartitions(self):
"""
return
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198590931
--- Diff: R/pkg/R/context.R ---
@@ -25,6 +25,22 @@ getMinPartitions <- function(sc, minPartitions) {
as.integer(minPartitions)
}
+#'
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198219194
--- Diff: R/pkg/R/context.R ---
@@ -25,6 +25,22 @@ getMinPartitions <- function(sc, minPartitions) {
as.integer(minPartitions)
}
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198174777
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala
---
@@ -446,6 +446,8 @@
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r197725870
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala
---
@@ -446,6 +446,8 @@
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r197120918
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala
---
@@ -446,6 +446,8 @@
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r197107564
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala
---
@@ -446,6 +446,8
GitHub user MaxGekk opened a pull request:
https://github.com/apache/spark/pull/21589
[SPARK-24591][CORE] Number of cores and executors in the cluster
## What changes were proposed in this pull request?
In the PR, I propose to extend `SparkContext` by:
1. `def
34 matches
Mail list logo
