Github user adrian555 commented on the issue:
https://github.com/apache/spark/pull/22455
Test was terminated as "Build execution time has reached the maximum
allowed time for your plan (90 minutes)."
Try to close and reopen the PR to see how the r
GitHub user adrian555 reopened a pull request:
https://github.com/apache/spark/pull/22455
[SPARK-24572][SPARKR] "eager execution" for R shell, IDE
## What changes were proposed in this pull request?
Check the `spark.sql.repl.eagerEval.enabled` configuratio
Github user adrian555 closed the pull request at:
https://github.com/apache/spark/pull/22455
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r227500368
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,52 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r227500418
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -3731,6 +3731,7 @@ test_that("catalog APIs, listTables, listColumns,
listFunc
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r227500321
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,52 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r227500277
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,52 @@
+#
--- End diff --
Done
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r227500208
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +246,33 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r226472697
--- Diff: R/pkg/R/DataFrame.R ---
@@ -246,30 +248,38 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r223257553
--- Diff: R/pkg/R/DataFrame.R ---
@@ -246,30 +248,38 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on the issue:
https://github.com/apache/spark/pull/22455
> I'm not sure about the new changes. IMO we are mixing S3 and S4
convention here as per @adrian555.
The implemented code is still conforming to S4 object standard that
`s
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r222451616
--- Diff: R/pkg/R/DataFrame.R ---
@@ -246,30 +248,38 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r222450798
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -21,11 +21,7 @@ context("Show SparkDataFrame when eager execution is
en
Github user adrian555 commented on the issue:
https://github.com/apache/spark/pull/22455
Thanks @falaki. We are taking out the `print` method out so the requirement
seems clear now. Yes, for Jupyter with IRkernel, two options
`jupyter.rich_display` and jupyter.base_display_func
Github user adrian555 commented on the issue:
https://github.com/apache/spark/pull/22455
Another flavor will be this
```r
print.SparkDataFrame <- function(x, ...) {
if ((length(list(...)) > 0) &&
identical(sparkR.conf("spark.sql.repl.eagerEval.en
Github user adrian555 commented on the issue:
https://github.com/apache/spark/pull/22455
@falaki trying to understand what you were asking for. I know that `print`
takes some arguments like `digit` and `quote` that can further format the
output. `print` is for S3 object and `show
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r220292594
--- Diff: docs/sparkr.md ---
@@ -450,6 +450,48 @@ print(model.summaries)
{% endhighlight %}
+### Eager execution
+
+If eager
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r220292560
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1491,9 +1491,10 @@ object SQLConf {
val
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r220292513
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +246,31 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r220292390
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,60 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on the issue:
https://github.com/apache/spark/pull/22455
Looks like the test was terminated in the middle, not likely related to the
any code change. Could someone please ask for retest? Thanks
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219971196
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +246,25 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219938610
--- Diff: docs/sparkr.md ---
@@ -450,6 +450,48 @@ print(model.summaries)
{% endhighlight %}
+### Eager execution
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219938566
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +246,25 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219937585
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,61 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219576011
--- Diff: R/pkg/R/DataFrame.R ---
@@ -226,7 +226,8 @@ setMethod("showDF",
#' show
#'
-#' Print class and type information
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219575988
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +245,15 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219384936
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +245,15 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219355827
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +245,15 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219351194
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +245,15 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219331649
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,58 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315429
--- Diff: R/pkg/R/DataFrame.R ---
@@ -244,11 +244,15 @@ setMethod("showDF",
#' @note show(SparkDataFrame) since 1.4.0
setMe
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315410
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,58 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315391
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,58 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315320
--- Diff: docs/sparkr.md ---
@@ -450,6 +450,42 @@ print(model.summaries)
{% endhighlight %}
+### Eager execution
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315374
--- Diff: R/pkg/tests/fulltests/test_eager_execution.R ---
@@ -0,0 +1,58 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315297
--- Diff: docs/sparkr.md ---
@@ -450,6 +450,42 @@ print(model.summaries)
{% endhighlight %}
+### Eager execution
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315251
--- Diff: docs/sparkr.md ---
@@ -450,6 +450,42 @@ print(model.summaries)
{% endhighlight %}
+### Eager execution
--- End diff
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22455#discussion_r219315277
--- Diff: docs/sparkr.md ---
@@ -450,6 +450,42 @@ print(model.summaries)
{% endhighlight %}
+### Eager execution
GitHub user adrian555 opened a pull request:
https://github.com/apache/spark/pull/22455
[SPARK-24572][SparkR] "eager execution" for R shell, IDE
## What changes were proposed in this pull request?
Check the `spark.sql.repl.eagerEval.enabled` configuratio
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9835#discussion_r45412567
--- Diff: R/pkg/R/utils.R ---
@@ -624,15 +624,22 @@ convertNamedListToEnv <- function(namedList) {
env
}
-# Assign a new environm
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9835#issuecomment-158228163
@felixcheung Yes, I agree. We should make this work for all dplyr-like
functions.
---
If your project is set up for it, you can reply to this email and have your
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9835#issuecomment-158220592
I was going to work on this JIRA but was busy on other deliverable. :)
Thanks @sun-rui
---
If your project is set up for it, you can reply to this email and have
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9835#discussion_r45410953
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1113,6 +1113,12 @@ setMethod("subset", signature(x = "DataFrame"),
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9835#discussion_r45413967
--- Diff: R/pkg/R/utils.R ---
@@ -624,15 +624,22 @@ convertNamedListToEnv <- function(namedList) {
env
}
-# Assign a new environm
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9443#discussion_r43930565
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2045,3 +2045,34 @@ setMethod("attach",
}
attach(newEnv, pos = pos, n
GitHub user adrian555 opened a pull request:
https://github.com/apache/spark/pull/9443
[SPARK-11260][SPARKR] with() function support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adrian555/spark with
Alternatively you can
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41698039
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9031#issuecomment-147096207
@sun-rui First, the current subsetting function '$' of DataFrame ends up
with a 'Column' class instead of a real column access of 'DataFrame'. So to
operate on one
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41701074
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9031#issuecomment-147146800
Hi admin,
The failure is
[error] (core/*:mimaPreviousClassfiles) sbt.ResolveException: download
failed: org.apache.spark#spark-unsafe_2.10;1.5.0!spark
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41701060
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41693561
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9031#issuecomment-147042281
@sun-rui The attach() function is a very useful R function and very
commonly used by customers. The link to the R function is
https://stat.ethz.ch/R-manual/R-devel
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41693574
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41693577
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41693517
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/9031#discussion_r41693523
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,33 @@ setMethod("as.data.frame",
}
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9031#issuecomment-146962533
@felixcheung Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/9031#issuecomment-146682793
Tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/8997#issuecomment-14559
@shivaram I have created the new pull request #9031 with the up-to-date
files.
---
If your project is set up for it, you can reply to this email and have your
reply
GitHub user adrian555 opened a pull request:
https://github.com/apache/spark/pull/9031
[SPARK-10913][SPARKR] attach() function support
Bring the change code up to date.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adrian555
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/8997#issuecomment-146671198
Thanks @shivaram. First time doing so. :) I will follow your suggestion
next time.
Closing this PR.
---
If your project is set up for it, you can reply
Github user adrian555 closed the pull request at:
https://github.com/apache/spark/pull/8997
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/8997#issuecomment-146269857
Hi, anything else needed from me to have this PR delivered? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear
GitHub user adrian555 opened a pull request:
https://github.com/apache/spark/pull/8997
[SPARK-10913][SPARKR] attach() function support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adrian555/spark attach1
Alternatively you
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/8985#issuecomment-145955245
I created a new pull request #8997 to address Shivaram's comment.
@felixcheung the R native detach() works so we do not need SparkR version
of detach
Github user adrian555 closed the pull request at:
https://github.com/apache/spark/pull/8985
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8997#discussion_r41319707
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,34 @@ setMethod("as.data.frame",
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8997#discussion_r4105
--- Diff: R/pkg/inst/tests/test_sparkSQL.R ---
@@ -1341,6 +1341,15 @@ test_that("Method as.data.frame as a synonym for
collect()", {
ex
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/8997#issuecomment-146027147
Hi admin, I have added some more test cases, could you please rerun? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8997#discussion_r41336776
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,34 @@ setMethod("as.data.frame",
Github user adrian555 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8997#discussion_r41336967
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1881,3 +1881,34 @@ setMethod("as.data.frame",
Github user adrian555 commented on the pull request:
https://github.com/apache/spark/pull/8985#issuecomment-145931079
Could admin please take a look at the error caught by jenkins run? It does
not seem to relate with any change from this PR.
---
If your project is set up for it, you
GitHub user adrian555 opened a pull request:
https://github.com/apache/spark/pull/8985
[SPARK-10913][SPARKR] attach() function support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adrian555/spark attach_and_with
75 matches
Mail list logo
