Github user junyangq commented on the issue:
https://github.com/apache/spark/pull/14705
@shivaram I found perhaps a neat way to document R'glm if we don't want to
remove it is to use `@inheritParams stats::glm`. That will bring in all the
parameters from `stats::glm` not listed in Spa
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/13796
@dbtsai Thanks for all of your meticulous review. Very much appreciated!
Glad we can have MLOR in Spark ML now.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14705
@HyukjinKwon - we don't have a coding style guide for R. We have some style
check with lint-r.
In addition, the document style you are looking at is a bit different from
coding style - this
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14705
@inheritParams would be the way to go.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429017
--- Diff: R/pkg/R/mllib.R ---
@@ -504,14 +504,15 @@ setMethod("summary", signature(object =
"IsotonicRegressionModel"),
#' Users can call \code{summary
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429158
--- Diff: R/pkg/R/mllib.R ---
@@ -917,14 +922,14 @@ setMethod("spark.lda", signature(data =
"SparkDataFrame"),
# Returns a summary of the AFT surviv
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14705
looking good - looks like we are very close.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14447
we are looking at establishing some guidelines in PR 14705. Let's hold on
for another day or 2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14639
**[Test build #64043 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64043/consoleFull)**
for PR 14639 at commit
[`fef88cd`](https://github.com/apache/spark/commit/f
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429434
--- Diff: R/pkg/R/generics.R ---
@@ -735,6 +752,8 @@ setGeneric("between", function(x, bounds) {
standardGeneric("between") })
setGeneric("cast", funct
Github user zjffdu commented on the issue:
https://github.com/apache/spark/pull/14639
Thanks @sun-rui, `EXISTING_SPARKR_BACKEND_PORT` do indicate cluster mode
indirectly for now. But here not only deployMode is unknown in R side, but also
master and other spark configurations. For n
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429536
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2464,8 +2489,10 @@ setMethod("unionAll",
#' Union two or more SparkDataFrames. This is equivalent to `UNION ALL` i
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429658
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3003,9 +3036,10 @@ setMethod("str",
#' Returns a new SparkDataFrame with columns dropped.
#' This is a no-op
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429664
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3187,6 +3221,7 @@ setMethod("histogram",
#' @param x A SparkDataFrame
#' @param url JDBC database url of the
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14705
@felixcheung Thanks for kind explanation. BTW, it'd be great too if it just
has a sentence, for example, `"For R code, Apache Spark follows lint-r"` in the
wiki just like Python has `"For Python
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14708
**[Test build #64034 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64034/consoleFull)**
for PR 14708 at commit
[`1e89cc3`](https://github.com/apache/spark/commit/
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75429772
--- Diff: R/pkg/R/mllib.R ---
@@ -620,11 +625,12 @@ setMethod("predict", signature(object =
"KMeansModel"),
#' predictions on new data, and \code{write
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14708
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user keypointt commented on the issue:
https://github.com/apache/spark/pull/14447
@felixcheung sure, no problem
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14708
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64034/
Test PASSed.
---
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14710
Can you put a more descriptive title for the change?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user sun-rui commented on the issue:
https://github.com/apache/spark/pull/14639
If in the future SparkConf is needed, instead of passing all spark conf to
R via env variables, we can expose API for accessing SparkConf in the R
backend, similar to that in Pyspark.
https://githu
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r7543
--- Diff: R/pkg/R/functions.R ---
@@ -3115,6 +3166,11 @@ setMethod("dense_rank",
#'
#' This is equivalent to the LAG function in SQL.
#'
+
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75430048
--- Diff: R/pkg/R/functions.R ---
@@ -3115,6 +3166,11 @@ setMethod("dense_rank",
#'
#' This is equivalent to the LAG function in SQL.
#'
+
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75430152
--- Diff: R/pkg/R/functions.R ---
@@ -1848,7 +1850,7 @@ setMethod("upper",
#' @note var since 1.6.0
setMethod("var",
signature(x = "
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14118
With this change, do all empty (e.g. zero sized string) values become null
values once they are read back?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75430253
--- Diff: R/pkg/R/SQLContext.R ---
@@ -727,6 +730,7 @@ dropTempView <- function(viewName) {
#' @param source The name of external data source
#' @p
GitHub user jagadeesanas2 opened a pull request:
https://github.com/apache/spark/pull/14711
[SPARK-16822] [DOC] [Support latex in scaladoc with MathJax]
## What changes were proposed in this pull request?
LaTeX is rendered as simple code, in `LinearRegression.scala`
```s
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14699
It's hard to say. Right now it is being converted on the [JVM
side](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala#L63)
- so it is
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14118
Also LGTM other than that major question.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
ena
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14711
**[Test build #64044 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64044/consoleFull)**
for PR 14711 at commit
[`7cacb11`](https://github.com/apache/spark/commit/7
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/8880
**[Test build #64035 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64035/consoleFull)**
for PR 8880 at commit
[`338210c`](https://github.com/apache/spark/commit/33
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14700
**[Test build #3226 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3226/consoleFull)**
for PR 14700 at commit
[`24bcf05`](https://github.com/apache/spark/commit/
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14709
I suspect array and struct literals will fail, looking at what Literal.sql
does. That said, it's an existing problem and we can fix that later.
---
If your project is set up for it, you can reply to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/8880
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64035/
Test PASSed.
---
I
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/8880
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
en
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14705
surely - we should have said `lint-r` as the baseline. There's definitely
more we could add though. It would be great if we have bandwidth to write more
[linters](https://github.com/jimhester/li
Github user angolon commented on the issue:
https://github.com/apache/spark/pull/14710
Done, sorry!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if t
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/14702
Can you update the description to say more about what this pr includes, and
what future todos are?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitH
Github user junyangq commented on a diff in the pull request:
https://github.com/apache/spark/pull/14705#discussion_r75431046
--- Diff: R/pkg/R/functions.R ---
@@ -319,7 +316,7 @@ setMethod("column",
#'
#' Computes the Pearson Correlation Coefficient for two Columns.
Github user jagadeesanas2 commented on the issue:
https://github.com/apache/spark/pull/14711
cc @lins05 @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishe
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14452
**[Test build #64037 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64037/consoleFull)**
for PR 14452 at commit
[`e094c14`](https://github.com/apache/spark/commit/
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431206
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +161,147 @@ private[spark] class HiveExternalCatalog(clien
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14452
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64037/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14452
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431369
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +163,172 @@ private[spark] class HiveExternalCatalog(clien
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431426
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -233,226 +229,21 @@ case class CreateDataSou
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/14452
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, o
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14452
**[Test build #64045 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64045/consoleFull)**
for PR 14452 at commit
[`e094c14`](https://github.com/apache/spark/commit/e
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/14702#discussion_r75431510
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ScriptTransformationExec.scala
---
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache So
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431528
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -200,22 +348,73 @@ private[spark] class HiveExternalCatalog(client
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431615
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +163,172 @@ private[spark] class HiveExternalCatalog(c
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14118
**[Test build #64040 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64040/consoleFull)**
for PR 14118 at commit
[`74b4dd8`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14118
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64040/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14118
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75431775
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
---
@@ -49,6 +49,8 @@ class MetastoreDataSourcesSuite extends Quer
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14709
**[Test build #64038 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64038/consoleFull)**
for PR 14709 at commit
[`a32cbc2`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14709
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14709
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64038/
Test PASSed.
---
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75432039
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
---
@@ -985,35 +987,37 @@ class MetastoreDataSourcesSuite extends
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75432071
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
---
@@ -49,6 +49,8 @@ class MetastoreDataSourcesSuite extends Quer
Github user zhengruifeng commented on the issue:
https://github.com/apache/spark/pull/14643
@srowen I though of `threshoulds` designed in ML just as a kind of
`weight`. This design is easy to understand. Is there some other librarys (like
sklearn) that support thresholds? We can refe
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75432051
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala
---
@@ -985,35 +987,37 @@ class MetastoreDataSourcesSuite extends
Github user sunLeung commented on the issue:
https://github.com/apache/spark/pull/4888
spark-submit on yarn did not distribute jars to nm-local-dir,How to fix
this problem?
Caused by: java.lang.IllegalStateException: Library directory
'/opt/hadoop/tmp/nm-local-dir/usercache/h
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75432300
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -144,16 +163,172 @@ private[spark] class HiveExternalCatalog(clien
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13152
**[Test build #3225 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3225/consoleFull)**
for PR 13152 at commit
[`9b8ce32`](https://github.com/apache/spark/commit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75432607
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -584,13 +579,8 @@ case class AlterTableSetLocationCommand(
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75432710
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -264,10 +261,8 @@ case class AlterTableUnsetPropertiesCommand(
GitHub user wzhfy opened a pull request:
https://github.com/apache/spark/pull/14712
[SPARK-17072] [SQL] support table-level statistics generation and storing
into/loading from metastore
## What changes were proposed in this pull request?
1. support generation table-level st
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/14155
Above is the data flow of all the fields in `CatalogTa
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14116
**[Test build #64041 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64041/consoleFull)**
for PR 14116 at commit
[`bd85aa5`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14712
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14116
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14116
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64041/
Test PASSed.
---
Github user scwf commented on the issue:
https://github.com/apache/spark/pull/14712
/cc @cloud-fan @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, o
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14181
**[Test build #64046 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64046/consoleFull)**
for PR 14181 at commit
[`b0f5dd5`](https://github.com/apache/spark/commit/b
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14705
**[Test build #64047 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64047/consoleFull)**
for PR 14705 at commit
[`870279a`](https://github.com/apache/spark/commit/8
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14181
**[Test build #64048 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64048/consoleFull)**
for PR 14181 at commit
[`c947583`](https://github.com/apache/spark/commit/c
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/14155#discussion_r75434165
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -200,22 +375,77 @@ private[spark] class HiveExternalCatalog(c
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14711
**[Test build #64044 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64044/consoleFull)**
for PR 14711 at commit
[`7cacb11`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14711
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64044/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14711
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/14118
@rxin yes all empty (e.g. zero sized string) values become null values once
they are read back.
E.g. given `test.csv`:
```
1,,3,
```
`spark.read.csv("test.csv").show()` produc
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14038
**[Test build #64049 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64049/consoleFull)**
for PR 14038 at commit
[`d53ad8e`](https://github.com/apache/spark/commit/d
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/14639
Are we talking about this
http://spark.apache.org/docs/latest/api/R/sparkR.conf.html?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as w
601 - 685 of 685 matches
Mail list logo
