[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179829#comment-16179829 ] Brendan Dwyer commented on SPARK-15799: --- I opened a [PR|https://github.com/apache/spark/pull/19342] that adds a {{SystemRequirements}} field and adds an {{on.exit()}} call after {{attach}} but if everyone is okay with using {{with}} I can close my PR. > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179747#comment-16179747 ] Brendan Dwyer commented on SPARK-15799: --- The {{Description}} field has been fixed. {{SystemRequirements}} is an easy 1 line fix. So I assume we need to discuses if we should change {{attach}} method to {{SparkDataFrame}} or appeal to the CRAN people for an exception. Thoughts? Here is the Good practice section referenced above: {quote} h2. Good practice {{attach}} has the side effect of altering the search path and this can easily lead to the wrong object of a particular name being found. People do often forget to {{detach}} databases. In interactive use, {{with}} is usually preferable to the use of attach/detach, unless what is a {{save()}}-produced file in which case {{attach()}} is a (safety) wrapper for {{load()}}. In programming, functions should not change the search path unless that is their purpose. Often {{with}} can be used within a function. If not, good practice is to * Always use a distinctive name argument, and * To immediately follow the attach call by an on.exit call to detach using the distinctive name. This ensures that the search path is left unchanged even if the function is interrupted or if code after the attach call changes the search path. {quote} > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114733#comment-16114733 ] Brendan Dwyer commented on SPARK-15799: --- [~felixcheung] awesome! Looking forward to this. > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113404#comment-16113404 ] Brendan Dwyer commented on SPARK-15799: --- Is there any update on this? > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21532) Improve console progress bar in RStudio
[ https://issues.apache.org/jira/browse/SPARK-21532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100806#comment-16100806 ] Brendan Dwyer commented on SPARK-21532: --- I've opened [an issue with RStudio|https://support.rstudio.com/hc/en-us/community/posts/115007135727-SparkR-progress-bar-issues]. Should I close this JIRA or keep it open until this is fixed? > Improve console progress bar in RStudio > --- > > Key: SPARK-21532 > URL: https://issues.apache.org/jira/browse/SPARK-21532 > Project: Spark > Issue Type: Improvement > Components: SparkR >Affects Versions: 2.2.0 > Environment: RStudio on Linux Mint, RStudio server on CentOS, > probably RStudio on other OS's >Reporter: Brendan Dwyer >Priority: Trivial > > When a SparkR job is run through terminal the progress bar appears neatly and > then disappears after the job is done however when a SparkR job is run in > RStudio the progress bar gets printed to multiple lines and never goes away: > {code} > [Stage 38:> (0 + 3) / > 8][Stage 38:===> (1 + 3) > / 8][Stage 38:=>(3 + > 3) / 8][Stage 38:=>(4 > + 3) / 8][Stage 38:===> > (6 + 2) / 8][Stage 38:==> > (7 + 1) / 8] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21532) Improve console progress bar in RStudio
[ https://issues.apache.org/jira/browse/SPARK-21532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100695#comment-16100695 ] Brendan Dwyer commented on SPARK-21532: --- https://support.rstudio.com/hc/en-us/community/posts/203136268-Carriage-return-does-not-work-on-stderr- > Improve console progress bar in RStudio > --- > > Key: SPARK-21532 > URL: https://issues.apache.org/jira/browse/SPARK-21532 > Project: Spark > Issue Type: Improvement > Components: SparkR >Affects Versions: 2.2.0 > Environment: RStudio on Linux Mint, RStudio server on CentOS, > probably RStudio on other OS's >Reporter: Brendan Dwyer >Priority: Trivial > > When a SparkR job is run through terminal the progress bar appears neatly and > then disappears after the job is done however when a SparkR job is run in > RStudio the progress bar gets printed to multiple lines and never goes away: > {code} > [Stage 38:> (0 + 3) / > 8][Stage 38:===> (1 + 3) > / 8][Stage 38:=>(3 + > 3) / 8][Stage 38:=>(4 > + 3) / 8][Stage 38:===> > (6 + 2) / 8][Stage 38:==> > (7 + 1) / 8] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21532) Improve console progress bar in RStudio
[ https://issues.apache.org/jira/browse/SPARK-21532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100693#comment-16100693 ] Brendan Dwyer commented on SPARK-21532: --- [~srowen] I assume so. I think in it's current state the progress bar is more annoying than it is useful in RStudio. Maybe we should set _spark.ui.showConsoleProgress_ to FALSE by default if the session is started from RStudio? > Improve console progress bar in RStudio > --- > > Key: SPARK-21532 > URL: https://issues.apache.org/jira/browse/SPARK-21532 > Project: Spark > Issue Type: Improvement > Components: SparkR >Affects Versions: 2.2.0 > Environment: RStudio on Linux Mint, RStudio server on CentOS, > probably RStudio on other OS's >Reporter: Brendan Dwyer >Priority: Trivial > > When a SparkR job is run through terminal the progress bar appears neatly and > then disappears after the job is done however when a SparkR job is run in > RStudio the progress bar gets printed to multiple lines and never goes away: > {code} > [Stage 38:> (0 + 3) / > 8][Stage 38:===> (1 + 3) > / 8][Stage 38:=>(3 + > 3) / 8][Stage 38:=>(4 > + 3) / 8][Stage 38:===> > (6 + 2) / 8][Stage 38:==> > (7 + 1) / 8] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-21532) Improve console progress bar in RStudio
Brendan Dwyer created SPARK-21532: - Summary: Improve console progress bar in RStudio Key: SPARK-21532 URL: https://issues.apache.org/jira/browse/SPARK-21532 Project: Spark Issue Type: Improvement Components: SparkR Affects Versions: 2.2.0 Environment: RStudio on Linux Mint, RStudio server on CentOS, probably RStudio on other OS's Reporter: Brendan Dwyer Priority: Trivial When a SparkR job is run through terminal the progress bar appears neatly and then disappears after the job is done however when a SparkR job is run in RStudio the progress bar gets printed to multiple lines and never goes away: {code} [Stage 38:> (0 + 3) / 8][Stage 38:===> (1 + 3) / 8][Stage 38:=>(3 + 3) / 8][Stage 38:=>(4 + 3) / 8][Stage 38:===> (6 + 2) / 8][Stage 38:==> (7 + 1) / 8] {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-20298) Spelling mistake: charactor
[ https://issues.apache.org/jira/browse/SPARK-20298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964741#comment-15964741 ] Brendan Dwyer commented on SPARK-20298: --- I can work on this > Spelling mistake: charactor > --- > > Key: SPARK-20298 > URL: https://issues.apache.org/jira/browse/SPARK-20298 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.1.0 >Reporter: Brendan Dwyer >Priority: Trivial > > "charactor" should be "character" > {code} > R/pkg/R/DataFrame.R:2821: stop("path should be charactor, NULL > or omitted.") > R/pkg/R/DataFrame.R:2828: stop("mode should be charactor or > omitted. It is 'error' by default.") > R/pkg/R/DataFrame.R:3043: stop("value should be an integer, > numeric, charactor or named list.") > R/pkg/R/DataFrame.R:3055: stop("Each item in value should be > an integer, numeric or charactor.") > R/pkg/R/DataFrame.R:3601: stop("outputMode should be charactor > or omitted.") > R/pkg/R/SQLContext.R:609:stop("path should be charactor, NULL or > omitted.") > R/pkg/inst/tests/testthat/test_sparkSQL.R:2929: "path should be > charactor, NULL or omitted.") > R/pkg/inst/tests/testthat/test_sparkSQL.R:2931: "mode should be > charactor or omitted. It is 'error' by default.") > R/pkg/inst/tests/testthat/test_sparkSQL.R:2950: "path should be > charactor, NULL or omitted.") > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-20298) Spelling mistake: charactor
Brendan Dwyer created SPARK-20298: - Summary: Spelling mistake: charactor Key: SPARK-20298 URL: https://issues.apache.org/jira/browse/SPARK-20298 Project: Spark Issue Type: Bug Components: SparkR Affects Versions: 2.1.0 Reporter: Brendan Dwyer Priority: Trivial "charactor" should be "character" {code} R/pkg/R/DataFrame.R:2821: stop("path should be charactor, NULL or omitted.") R/pkg/R/DataFrame.R:2828: stop("mode should be charactor or omitted. It is 'error' by default.") R/pkg/R/DataFrame.R:3043: stop("value should be an integer, numeric, charactor or named list.") R/pkg/R/DataFrame.R:3055: stop("Each item in value should be an integer, numeric or charactor.") R/pkg/R/DataFrame.R:3601: stop("outputMode should be charactor or omitted.") R/pkg/R/SQLContext.R:609:stop("path should be charactor, NULL or omitted.") R/pkg/inst/tests/testthat/test_sparkSQL.R:2929: "path should be charactor, NULL or omitted.") R/pkg/inst/tests/testthat/test_sparkSQL.R:2931: "mode should be charactor or omitted. It is 'error' by default.") R/pkg/inst/tests/testthat/test_sparkSQL.R:2950: "path should be charactor, NULL or omitted.") {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19387) CRAN tests do not run with SparkR source package
[ https://issues.apache.org/jira/browse/SPARK-19387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843493#comment-15843493 ] Brendan Dwyer commented on SPARK-19387: --- Should this be under [SPARK-15799]? > CRAN tests do not run with SparkR source package > > > Key: SPARK-19387 > URL: https://issues.apache.org/jira/browse/SPARK-19387 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.1.0 >Reporter: Felix Cheung >Assignee: Felix Cheung > > It looks like sparkR.session() is not installing Spark - as a result, running > R CMD check --as-cran SparkR_*.tar.gz fails, blocking possible submission to > CRAN. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753093#comment-15753093 ] Brendan Dwyer edited comment on SPARK-18817 at 12/16/16 1:45 AM: - I'm not sure the CRAN people would be okay with that. It might be enough to pass any automatic testing they have but it would still be against their policies. {quote} Limited exceptions may be allowed in interactive sessions if the package *obtains confirmation from the user*. {quote} was (Author: bdwyer): I'm not sure the CRAN people would be okay with that. It might be enough to pass any automatic testing they have but it would still be against their policies. > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer >Priority: Critical > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753093#comment-15753093 ] Brendan Dwyer edited comment on SPARK-18817 at 12/16/16 1:30 AM: - I'm not sure the CRAN people would be okay with that. It might be enough to pass any automatic testing they have but it would still be against their policies. was (Author: bdwyer): I'm not sure the CRAN people would be okay with that. > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer >Priority: Critical > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753095#comment-15753095 ] Brendan Dwyer commented on SPARK-18817: --- {code} library("SparkR") sparkR.session() df <- as.DataFrame(iris) {code} > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer >Priority: Critical > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753093#comment-15753093 ] Brendan Dwyer commented on SPARK-18817: --- I'm not sure the CRAN people would be okay with that. > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer >Priority: Critical > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752951#comment-15752951 ] Brendan Dwyer commented on SPARK-18817: --- [~shivaram] it does not happen if I disable Hive. > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer >Priority: Critical > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15752844#comment-15752844 ] Brendan Dwyer commented on SPARK-18817: --- I'm also seeing _derby.log_ and a folder named _metastore_db_ being created in my working directory when I run the following: {code} library("SparkR") sparkR.session() df <- as.DataFrame(iris) {code} > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer >Priority: Critical > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brendan Dwyer updated SPARK-18817: -- Description: Per CRAN policies https://cran.r-project.org/web/packages/policies.html {quote} - Packages should not write in the users’ home filespace, nor anywhere else on the file system apart from the R session’s temporary directory (or during installation in the location pointed to by TMPDIR: and such usage should be cleaned up). Installing into the system’s R installation (e.g., scripts to its bin directory) is not allowed. Limited exceptions may be allowed in interactive sessions if the package obtains confirmation from the user. - Packages should not modify the global environment (user’s workspace). {quote} Currently "spark-warehouse" gets created in the working directory when sparkR.session() is called. was: Per CRAN policies https://cran.r-project.org/web/packages/policies.html "Packages should not write in the users’ home filespace, nor anywhere else on the file system apart from the R session’s temporary directory (or during installation in the location pointed to by TMPDIR: and such usage should be cleaned up). Installing into the system’s R installation (e.g., scripts to its bin directory) is not allowed. Limited exceptions may be allowed in interactive sessions if the package obtains confirmation from the user. - Packages should not modify the global environment (user’s workspace)." Currently "spark-warehouse" gets created in the working directory when sparkR.session() is called. > Ensure nothing is written outside R's tempdir() by default > -- > > Key: SPARK-18817 > URL: https://issues.apache.org/jira/browse/SPARK-18817 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Brendan Dwyer > > Per CRAN policies > https://cran.r-project.org/web/packages/policies.html > {quote} > - Packages should not write in the users’ home filespace, nor anywhere else > on the file system apart from the R session’s temporary directory (or during > installation in the location pointed to by TMPDIR: and such usage should be > cleaned up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > Limited exceptions may be allowed in interactive sessions if the package > obtains confirmation from the user. > - Packages should not modify the global environment (user’s workspace). > {quote} > Currently "spark-warehouse" gets created in the working directory when > sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default
Brendan Dwyer created SPARK-18817: - Summary: Ensure nothing is written outside R's tempdir() by default Key: SPARK-18817 URL: https://issues.apache.org/jira/browse/SPARK-18817 Project: Spark Issue Type: Sub-task Components: SparkR Reporter: Brendan Dwyer Per CRAN policies https://cran.r-project.org/web/packages/policies.html "Packages should not write in the users’ home filespace, nor anywhere else on the file system apart from the R session’s temporary directory (or during installation in the location pointed to by TMPDIR: and such usage should be cleaned up). Installing into the system’s R installation (e.g., scripts to its bin directory) is not allowed. Limited exceptions may be allowed in interactive sessions if the package obtains confirmation from the user. - Packages should not modify the global environment (user’s workspace)." Currently "spark-warehouse" gets created in the working directory when sparkR.session() is called. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736657#comment-15736657 ] Brendan Dwyer commented on SPARK-15799: --- I think that's the best solution. Can I create a pull request to make this change? > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736591#comment-15736591 ] Brendan Dwyer commented on SPARK-15799: --- On my machine it gets created in my home directory when I call sparkR.session() > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15799) Release SparkR on CRAN
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736572#comment-15736572 ] Brendan Dwyer commented on SPARK-15799: --- [CRAN policy|https://cran.r-project.org/web/packages/policies.html] states: {quote} - Packages should not write in the users’ home filespace, nor anywhere else on the file system apart from the R session’s temporary directory (or during installation in the location pointed to by TMPDIR: and such usage should be cleaned up). Installing into the system’s R installation (e.g., scripts to its bin directory) is not allowed. Limited exceptions may be allowed in interactive sessions if the package obtains confirmation from the user. - Packages should not modify the global environment (user’s workspace). {quote} Do we need to move the location of spark-warehouse to a temporary directory? > Release SparkR on CRAN > -- > > Key: SPARK-15799 > URL: https://issues.apache.org/jira/browse/SPARK-15799 > Project: Spark > Issue Type: New Feature > Components: SparkR >Reporter: Xiangrui Meng > > Story: "As an R user, I would like to see SparkR released on CRAN, so I can > use SparkR easily in an existing R environment and have other packages built > on top of SparkR." > I made this JIRA with the following questions in mind: > * Are there known issues that prevent us releasing SparkR on CRAN? > * Do we want to package Spark jars in the SparkR release? > * Are there license issues? > * How does it fit into Spark's release process? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org