Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.jdbc.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.jdbc.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.jdbc.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,105 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Create a SparkDataFrame representing the database table...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for read.jdbc {SparkR}"><tr><td>read.jdbc {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Create a SparkDataFrame representing the database table accessible via JDBC URL</h2> + +<h3>Description</h3> + +<p>Additional JDBC database connection properties can be set (...) +</p> + + +<h3>Usage</h3> + +<pre> +read.jdbc(url, tableName, partitionColumn = NULL, lowerBound = NULL, + upperBound = NULL, numPartitions = 0L, predicates = list(), ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>url</code></td> +<td> +<p>JDBC database url of the form <code>jdbc:subprotocol:subname</code></p> +</td></tr> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>the name of the table in the external database</p> +</td></tr> +<tr valign="top"><td><code>partitionColumn</code></td> +<td> +<p>the name of a column of integral type that will be used for partitioning</p> +</td></tr> +<tr valign="top"><td><code>lowerBound</code></td> +<td> +<p>the minimum value of <code>partitionColumn</code> used to decide partition stride</p> +</td></tr> +<tr valign="top"><td><code>upperBound</code></td> +<td> +<p>the maximum value of <code>partitionColumn</code> used to decide partition stride</p> +</td></tr> +<tr valign="top"><td><code>numPartitions</code></td> +<td> +<p>the number of partitions, This, along with <code>lowerBound</code> (inclusive), +<code>upperBound</code> (exclusive), form partition strides for generated WHERE +clause expressions used to split the column <code>partitionColumn</code> evenly. +This defaults to SparkContext.defaultParallelism when unset.</p> +</td></tr> +<tr valign="top"><td><code>predicates</code></td> +<td> +<p>a list of conditions in the where clause; each one defines one partition</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional JDBC database connection named properties.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Only one of partitionColumn or predicates should be set. Partitions of the table will be +retrieved in parallel based on the <code>numPartitions</code> or by the predicates. +</p> +<p>Don't create too many partitions in parallel on a large cluster; otherwise Spark might crash +your external database systems. +</p> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.jdbc since 2.0.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D jdbcUrl <- "jdbc:mysql://localhost:3306/databasename" +##D df <- read.jdbc(jdbcUrl, "table", predicates = list("field<=123"), user = "username") +##D df2 <- read.jdbc(jdbcUrl, "table2", partitionColumn = "index", lowerBound = 0, +##D upperBound = 10000, user = "username", password = "password") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html>
Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.json.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.json.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.json.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,77 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Create a SparkDataFrame from a JSON file.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for read.json {SparkR}"><tr><td>read.json {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Create a SparkDataFrame from a JSON file.</h2> + +<h3>Description</h3> + +<p>Loads a JSON file, returning the result as a SparkDataFrame +By default, (<a href="http://jsonlines.org/">JSON Lines text format or newline-delimited JSON</a> +) is supported. For JSON (one record per file), set a named property <code>multiLine</code> to +<code>TRUE</code>. +It goes through the entire dataset once to determine the schema. +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +read.json(path, ...) + +## Default S3 method: +jsonFile(path) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>Path of file to read. A vector of multiple paths is allowed.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional external data source specific named properties.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.json since 1.6.0 +</p> +<p>jsonFile since 1.4.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D df <- read.json(path, multiLine = TRUE) +##D df <- jsonFile(path) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.ml.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.ml.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.ml.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,66 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Load a fitted MLlib model from the input path.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for read.ml {SparkR}"><tr><td>read.ml {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Load a fitted MLlib model from the input path.</h2> + +<h3>Description</h3> + +<p>Load a fitted MLlib model from the input path. +</p> + + +<h3>Usage</h3> + +<pre> +read.ml(path) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>path of the model to read.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A fitted MLlib model. +</p> + + +<h3>Note</h3> + +<p>read.ml since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p><a href="write.ml.html">write.ml</a> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D path <- "path/to/model" +##D model <- read.ml(path) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.orc.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.orc.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.orc.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,49 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Create a SparkDataFrame from an ORC file.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for read.orc {SparkR}"><tr><td>read.orc {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Create a SparkDataFrame from an ORC file.</h2> + +<h3>Description</h3> + +<p>Loads an ORC file, returning the result as a SparkDataFrame. +</p> + + +<h3>Usage</h3> + +<pre> +read.orc(path, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>Path of file to read.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional external data source specific named properties.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.orc since 2.0.0 +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.parquet.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.parquet.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.parquet.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,55 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Create a SparkDataFrame from a Parquet file.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for read.parquet {SparkR}"><tr><td>read.parquet {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Create a SparkDataFrame from a Parquet file.</h2> + +<h3>Description</h3> + +<p>Loads a Parquet file, returning the result as a SparkDataFrame. +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +read.parquet(path, ...) + +## Default S3 method: +parquetFile(...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>path of file to read. A vector of multiple paths is allowed.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>argument(s) passed to the method.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.parquet since 1.6.0 +</p> +<p>parquetFile since 1.4.0 +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.stream.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.stream.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.stream.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,93 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Load a streaming SparkDataFrame</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for read.stream {SparkR}"><tr><td>read.stream {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Load a streaming SparkDataFrame</h2> + +<h3>Description</h3> + +<p>Returns the dataset in a data source as a SparkDataFrame +</p> + + +<h3>Usage</h3> + +<pre> +read.stream(source = NULL, schema = NULL, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>source</code></td> +<td> +<p>The name of external data source</p> +</td></tr> +<tr valign="top"><td><code>schema</code></td> +<td> +<p>The data schema defined in structType or a DDL-formatted string, this is +required for file-based streaming data source</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional external data source specific named options, for instance <code>path</code> for +file-based streaming data source. <code>timeZone</code> to indicate a timezone to be used to +parse timestamps in the JSON/CSV data sources or partition values; If it isn't set, it +uses the default value, session local timezone.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>The data source is specified by the <code>source</code> and a set of options(...). +If <code>source</code> is not specified, the default data source configured by +"spark.sql.sources.default" will be used. +</p> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.stream since 2.2.0 +</p> +<p>experimental +</p> + + +<h3>See Also</h3> + +<p><a href="write.stream.html">write.stream</a> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D df <- read.stream("socket", host = "localhost", port = 9999) +##D q <- write.stream(df, "text", path = "/home/user/out", checkpointLocation = "/home/user/cp") +##D +##D df <- read.stream("json", path = jsonDir, schema = schema, maxFilesPerTrigger = 1) +##D stringSchema <- "name STRING, info MAP<STRING, DOUBLE>" +##D df1 <- read.stream("json", path = jsonDir, schema = stringSchema, maxFilesPerTrigger = 1) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.text.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.text.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.text.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,74 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Create a SparkDataFrame from a text file.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for read.text {SparkR}"><tr><td>read.text {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Create a SparkDataFrame from a text file.</h2> + +<h3>Description</h3> + +<p>Loads text files and returns a SparkDataFrame whose schema starts with +a string column named "value", and followed by partitioned columns if +there are any. +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +read.text(path, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>Path of file to read. A vector of multiple paths is allowed.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional external data source specific named properties.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Each line in the text file is a new row in the resulting SparkDataFrame. +</p> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.text since 1.6.1 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.txt" +##D df <- read.text(path) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/recoverPartitions.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/recoverPartitions.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/recoverPartitions.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,56 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Recovers all the partitions in the directory of a table and...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for recoverPartitions {SparkR}"><tr><td>recoverPartitions {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Recovers all the partitions in the directory of a table and update the catalog</h2> + +<h3>Description</h3> + +<p>Recovers all the partitions in the directory of a table and update the catalog. The name should +reference a partitioned table, and not a view. +</p> + + +<h3>Usage</h3> + +<pre> +recoverPartitions(tableName) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>the qualified or unqualified name that designates a table. If no database +identifier is provided, it refers to a table in the current database.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>since 2.2.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D recoverPartitions("myTable") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/refreshByPath.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/refreshByPath.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/refreshByPath.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,56 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Invalidates and refreshes all the cached data and metadata...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for refreshByPath {SparkR}"><tr><td>refreshByPath {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Invalidates and refreshes all the cached data and metadata for SparkDataFrame containing path</h2> + +<h3>Description</h3> + +<p>Invalidates and refreshes all the cached data (and the associated metadata) for any +SparkDataFrame that contains the given data source path. Path matching is by prefix, i.e. "/" +would invalidate everything that is cached. +</p> + + +<h3>Usage</h3> + +<pre> +refreshByPath(path) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>the path of the data source.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>since 2.2.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D refreshByPath("/path") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/refreshTable.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/refreshTable.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/refreshTable.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,65 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Invalidates and refreshes all the cached data and metadata of...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for refreshTable {SparkR}"><tr><td>refreshTable {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Invalidates and refreshes all the cached data and metadata of the given table</h2> + +<h3>Description</h3> + +<p>Invalidates and refreshes all the cached data and metadata of the given table. For performance +reasons, Spark SQL or the external data source library it uses might cache certain metadata about +a table, such as the location of blocks. When those change outside of Spark SQL, users should +call this function to invalidate the cache. +</p> + + +<h3>Usage</h3> + +<pre> +refreshTable(tableName) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>the qualified or unqualified name that designates a table. If no database +identifier is provided, it refers to a table in the current database.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>If this table is cached as an InMemoryRelation, drop the original cached version and make the +new version cached lazily. +</p> + + +<h3>Note</h3> + +<p>since 2.2.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D refreshTable("myTable") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/registerTempTable-deprecated.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/registerTempTable-deprecated.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/registerTempTable-deprecated.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,113 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: (Deprecated) Register Temporary Table</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for registerTempTable {SparkR}"><tr><td>registerTempTable {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>(Deprecated) Register Temporary Table</h2> + +<h3>Description</h3> + +<p>Registers a SparkDataFrame as a Temporary Table in the SparkSession +</p> + + +<h3>Usage</h3> + +<pre> +registerTempTable(x, tableName) + +## S4 method for signature 'SparkDataFrame,character' +registerTempTable(x, tableName) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>A character vector containing the name of the table</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>registerTempTable since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p><a href="createOrReplaceTempView.html">createOrReplaceTempView</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D registerTempTable(df, "json_df") +##D new_df <- sql("SELECT * FROM json_df") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/rename.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/rename.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/rename.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,140 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: rename</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for rename {SparkR}"><tr><td>rename {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>rename</h2> + +<h3>Description</h3> + +<p>Rename an existing column in a SparkDataFrame. +</p> + + +<h3>Usage</h3> + +<pre> +rename(x, ...) + +withColumnRenamed(x, existingCol, newCol) + +## S4 method for signature 'SparkDataFrame,character,character' +withColumnRenamed(x, existingCol, + newCol) + +## S4 method for signature 'SparkDataFrame' +rename(x, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>A named pair of the form new_column_name = existing_column</p> +</td></tr> +<tr valign="top"><td><code>existingCol</code></td> +<td> +<p>The name of the column you want to change.</p> +</td></tr> +<tr valign="top"><td><code>newCol</code></td> +<td> +<p>The new column name.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A SparkDataFrame with the column name changed. +</p> + + +<h3>Note</h3> + +<p>withColumnRenamed since 1.4.0 +</p> +<p>rename since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p><a href="mutate.html">mutate</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D newDF <- withColumnRenamed(df, "col1", "newCol1") +## End(Not run) +## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D newDF <- rename(df, col1 = df$newCol1) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/repartition.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/repartition.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/repartition.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,138 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Repartition</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for repartition {SparkR}"><tr><td>repartition {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Repartition</h2> + +<h3>Description</h3> + +<p>The following options for repartition are possible: +</p> + +<ul> +<li><p>1. Return a new SparkDataFrame that has exactly <code>numPartitions</code>. +</p> +</li> +<li><p>2. Return a new SparkDataFrame hash partitioned by +the given columns into <code>numPartitions</code>. +</p> +</li> +<li><p>3. Return a new SparkDataFrame hash partitioned by the given column(s), +using <code>spark.sql.shuffle.partitions</code> as number of partitions. +</p> +</li></ul> + + + +<h3>Usage</h3> + +<pre> +repartition(x, ...) + +## S4 method for signature 'SparkDataFrame' +repartition(x, numPartitions = NULL, col = NULL, + ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional column(s) to be used in the partitioning.</p> +</td></tr> +<tr valign="top"><td><code>numPartitions</code></td> +<td> +<p>the number of partitions to use.</p> +</td></tr> +<tr valign="top"><td><code>col</code></td> +<td> +<p>the column by which the partitioning will be performed.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>repartition since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p><a href="coalesce.html">coalesce</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D newDF <- repartition(df, 2L) +##D newDF <- repartition(df, numPartitions = 2L) +##D newDF <- repartition(df, col = df$"col1", df$"col2") +##D newDF <- repartition(df, 3L, col = df$"col1", df$"col2") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/rollup.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/rollup.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/rollup.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,127 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: rollup</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for rollup {SparkR}"><tr><td>rollup {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>rollup</h2> + +<h3>Description</h3> + +<p>Create a multi-dimensional rollup for the SparkDataFrame using the specified columns. +</p> + + +<h3>Usage</h3> + +<pre> +rollup(x, ...) + +## S4 method for signature 'SparkDataFrame' +rollup(x, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>character name(s) or Column(s) to group on.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>If grouping expression is missing <code>rollup</code> creates a single global aggregate and is +equivalent to direct application of <a href="summarize.html">agg</a>. +</p> + + +<h3>Value</h3> + +<p>A GroupedData. +</p> + + +<h3>Note</h3> + +<p>rollup since 2.3.0 +</p> + + +<h3>See Also</h3> + +<p><a href="summarize.html">agg</a>, <a href="cube.html">cube</a>, <a href="groupBy.html">groupBy</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D df <- createDataFrame(mtcars) +##D mean(rollup(df, "cyl", "gear", "am"), "mpg") +##D +##D # Following calls are equivalent +##D agg(rollup(df), mean(df$mpg)) +##D agg(df, mean(df$mpg)) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/rowsBetween.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/rowsBetween.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/rowsBetween.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,87 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: rowsBetween</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for rowsBetween {SparkR}"><tr><td>rowsBetween {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>rowsBetween</h2> + +<h3>Description</h3> + +<p>Defines the frame boundaries, from <code>start</code> (inclusive) to <code>end</code> (inclusive). +</p> + + +<h3>Usage</h3> + +<pre> +rowsBetween(x, start, end) + +## S4 method for signature 'WindowSpec,numeric,numeric' +rowsBetween(x, start, end) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a WindowSpec</p> +</td></tr> +<tr valign="top"><td><code>start</code></td> +<td> +<p>boundary start, inclusive. +The frame is unbounded if this is the minimum long value.</p> +</td></tr> +<tr valign="top"><td><code>end</code></td> +<td> +<p>boundary end, inclusive. +The frame is unbounded if this is the maximum long value.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Both <code>start</code> and <code>end</code> are relative positions from the current row. For example, +"0" means "current row", while "-1" means the row before the current row, and "5" means the +fifth row after the current row. +</p> + + +<h3>Value</h3> + +<p>a WindowSpec +</p> + + +<h3>Note</h3> + +<p>rowsBetween since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p>Other windowspec_method: <code><a href="orderBy.html">orderBy</a></code>, +<code><a href="partitionBy.html">partitionBy</a></code>, <code><a href="rangeBetween.html">rangeBetween</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D rowsBetween(ws, 0, 3) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/sample.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/sample.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/sample.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,130 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Sample</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for sample {SparkR}"><tr><td>sample {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Sample</h2> + +<h3>Description</h3> + +<p>Return a sampled subset of this SparkDataFrame using a random seed. +Note: this is not guaranteed to provide exactly the fraction specified +of the total count of of the given SparkDataFrame. +</p> + + +<h3>Usage</h3> + +<pre> +sample(x, withReplacement = FALSE, fraction, seed) + +sample_frac(x, withReplacement = FALSE, fraction, seed) + +## S4 method for signature 'SparkDataFrame' +sample(x, withReplacement = FALSE, fraction, seed) + +## S4 method for signature 'SparkDataFrame' +sample_frac(x, withReplacement = FALSE, fraction, + seed) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>withReplacement</code></td> +<td> +<p>Sampling with replacement or not</p> +</td></tr> +<tr valign="top"><td><code>fraction</code></td> +<td> +<p>The (rough) sample target fraction</p> +</td></tr> +<tr valign="top"><td><code>seed</code></td> +<td> +<p>Randomness seed value. Default is a random seed.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>sample since 1.4.0 +</p> +<p>sample_frac since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D collect(sample(df, fraction = 0.5)) +##D collect(sample(df, FALSE, 0.5)) +##D collect(sample(df, TRUE, 0.5, seed = 3)) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/sampleBy.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/sampleBy.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/sampleBy.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,86 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Returns a stratified sample without replacement</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for sampleBy {SparkR}"><tr><td>sampleBy {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Returns a stratified sample without replacement</h2> + +<h3>Description</h3> + +<p>Returns a stratified sample without replacement based on the fraction given on each +stratum. +</p> + + +<h3>Usage</h3> + +<pre> +sampleBy(x, col, fractions, seed) + +## S4 method for signature 'SparkDataFrame,character,list,numeric' +sampleBy(x, col, fractions, + seed) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>col</code></td> +<td> +<p>column that defines strata</p> +</td></tr> +<tr valign="top"><td><code>fractions</code></td> +<td> +<p>A named list giving sampling fraction for each stratum. If a stratum is +not specified, we treat its fraction as zero.</p> +</td></tr> +<tr valign="top"><td><code>seed</code></td> +<td> +<p>random seed</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A new SparkDataFrame that represents the stratified sample +</p> + + +<h3>Note</h3> + +<p>sampleBy since 1.6.0 +</p> + + +<h3>See Also</h3> + +<p>Other stat functions: <code><a href="approxQuantile.html">approxQuantile</a></code>, +<code><a href="corr.html">corr</a></code>, <code><a href="cov.html">cov</a></code>, +<code><a href="crosstab.html">crosstab</a></code>, <code><a href="freqItems.html">freqItems</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D df <- read.json("/path/to/file.json") +##D sample <- sampleBy(df, "key", fractions, 36) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/saveAsTable.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/saveAsTable.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/saveAsTable.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,139 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Save the contents of the SparkDataFrame to a data source as a...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for saveAsTable {SparkR}"><tr><td>saveAsTable {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Save the contents of the SparkDataFrame to a data source as a table</h2> + +<h3>Description</h3> + +<p>The data source is specified by the <code>source</code> and a set of options (...). +If <code>source</code> is not specified, the default data source configured by +spark.sql.sources.default will be used. +</p> + + +<h3>Usage</h3> + +<pre> +saveAsTable(df, tableName, source = NULL, mode = "error", ...) + +## S4 method for signature 'SparkDataFrame,character' +saveAsTable(df, tableName, source = NULL, + mode = "error", ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>df</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>a name for the table.</p> +</td></tr> +<tr valign="top"><td><code>source</code></td> +<td> +<p>a name for external data source.</p> +</td></tr> +<tr valign="top"><td><code>mode</code></td> +<td> +<p>one of 'append', 'overwrite', 'error', 'errorifexists', 'ignore' +save mode (it is 'error' by default)</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional option(s) passed to the method.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Additionally, mode is used to specify the behavior of the save operation when +data already exists in the data source. There are four modes: <br /> +'append': Contents of this SparkDataFrame are expected to be appended to existing data. <br /> +'overwrite': Existing data is expected to be overwritten by the contents of this +SparkDataFrame. <br /> +'error' or 'errorifexists': An exception is expected to be thrown. <br /> +'ignore': The save operation is expected to not save the contents of the SparkDataFrame +and to not change the existing data. <br /> +</p> + + +<h3>Note</h3> + +<p>saveAsTable since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D saveAsTable(df, "myfile") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/schema.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/schema.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/schema.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,106 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Get schema object</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for schema {SparkR}"><tr><td>schema {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Get schema object</h2> + +<h3>Description</h3> + +<p>Returns the schema of this SparkDataFrame as a structType object. +</p> + + +<h3>Usage</h3> + +<pre> +schema(x) + +## S4 method for signature 'SparkDataFrame' +schema(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>schema since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D dfSchema <- schema(df) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/select.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/select.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/select.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,157 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Select</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for select {SparkR}"><tr><td>select {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Select</h2> + +<h3>Description</h3> + +<p>Selects a set of columns with names or Column expressions. +</p> + + +<h3>Usage</h3> + +<pre> +select(x, col, ...) + +## S4 method for signature 'SparkDataFrame' +x$name + +## S4 replacement method for signature 'SparkDataFrame' +x$name <- value + +## S4 method for signature 'SparkDataFrame,character' +select(x, col, ...) + +## S4 method for signature 'SparkDataFrame,Column' +select(x, col, ...) + +## S4 method for signature 'SparkDataFrame,list' +select(x, col) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>col</code></td> +<td> +<p>a list of columns or single Column or name.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional column(s) if only one column is specified in <code>col</code>. +If more than one column is assigned in <code>col</code>, <code>...</code> +should be left empty.</p> +</td></tr> +<tr valign="top"><td><code>name</code></td> +<td> +<p>name of a Column (without being wrapped by <code>""</code>).</p> +</td></tr> +<tr valign="top"><td><code>value</code></td> +<td> +<p>a Column or an atomic vector in the length of 1 as literal value, or <code>NULL</code>. +If <code>NULL</code>, the specified Column is dropped.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A new SparkDataFrame with selected columns. +</p> + + +<h3>Note</h3> + +<p>$ since 1.4.0 +</p> +<p>$<- since 1.4.0 +</p> +<p>select(SparkDataFrame, character) since 1.4.0 +</p> +<p>select(SparkDataFrame, Column) since 1.4.0 +</p> +<p>select(SparkDataFrame, list) since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> +<p>Other subsetting functions: <code><a href="filter.html">filter</a></code>, +<code><a href="subset.html">subset</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D select(df, "*") +##D select(df, "col1", "col2") +##D select(df, df$name, df$age + 1) +##D select(df, c("col1", "col2")) +##D select(df, list(df$name, df$age + 1)) +##D # Similar to R data frames columns can also be selected using $ +##D df[,df$age] +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org