http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/summary.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/summary.html b/site/docs/2.3.0/api/R/summary.html new file mode 100644 index 0000000..21fbff6 --- /dev/null +++ b/site/docs/2.3.0/api/R/summary.html @@ -0,0 +1,150 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: summary</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for summary {SparkR}"><tr><td>summary {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>summary</h2> + +<h3>Description</h3> + +<p>Computes specified statistics for numeric and string columns. Available statistics are: +</p> + +<ul> +<li><p> count +</p> +</li> +<li><p> mean +</p> +</li> +<li><p> stddev +</p> +</li> +<li><p> min +</p> +</li> +<li><p> max +</p> +</li> +<li><p> arbitrary approximate percentiles specified as a percentage (eg, "75%") +</p> +</li></ul> + +<p>If no statistics are given, this function computes count, mean, stddev, min, +approximate quartiles (percentiles at 25%, 50%, and 75%), and max. +This function is meant for exploratory data analysis, as we make no guarantee about the +backward compatibility of the schema of the resulting Dataset. If you want to +programmatically compute summary statistics, use the <code>agg</code> function instead. +</p> + + +<h3>Usage</h3> + +<pre> +summary(object, ...) + +## S4 method for signature 'SparkDataFrame' +summary(object, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>object</code></td> +<td> +<p>a SparkDataFrame to be summarized.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>(optional) statistics to be computed for all columns.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A SparkDataFrame. +</p> + + +<h3>Note</h3> + +<p>summary(SparkDataFrame) since 1.5.0 +</p> +<p>The statistics provided by <code>summary</code> were change in 2.3.0 use <a href="describe.html">describe</a> for +previous defaults. +</p> + + +<h3>See Also</h3> + +<p><a href="describe.html">describe</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D summary(df) +##D summary(df, "min", "25%", "75%", "max") +##D summary(select(df, "age", "height")) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html>
http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/tableNames.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/tableNames.html b/site/docs/2.3.0/api/R/tableNames.html new file mode 100644 index 0000000..24f5429 --- /dev/null +++ b/site/docs/2.3.0/api/R/tableNames.html @@ -0,0 +1,61 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Table Names</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for tableNames {SparkR}"><tr><td>tableNames {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Table Names</h2> + +<h3>Description</h3> + +<p>Returns the names of tables in the given database as an array. +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +tableNames(databaseName = NULL) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>databaseName</code></td> +<td> +<p>(optional) name of the database</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>a list of table names +</p> + + +<h3>Note</h3> + +<p>tableNames since 1.4.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D tableNames("hive") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/tableToDF.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/tableToDF.html b/site/docs/2.3.0/api/R/tableToDF.html new file mode 100644 index 0000000..ba795c6 --- /dev/null +++ b/site/docs/2.3.0/api/R/tableToDF.html @@ -0,0 +1,67 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Create a SparkDataFrame from a SparkSQL table or view</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for tableToDF {SparkR}"><tr><td>tableToDF {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Create a SparkDataFrame from a SparkSQL table or view</h2> + +<h3>Description</h3> + +<p>Returns the specified table or view as a SparkDataFrame. The table or view must already exist or +have already been registered in the SparkSession. +</p> + + +<h3>Usage</h3> + +<pre> +tableToDF(tableName) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>the qualified or unqualified name that designates a table or view. If a database +is specified, it identifies the table/view from the database. +Otherwise, it first attempts to find a temporary view with the given name +and then match the table/view from the current database.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>tableToDF since 2.0.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D createOrReplaceTempView(df, "table") +##D new_df <- tableToDF("table") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/tables.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/tables.html b/site/docs/2.3.0/api/R/tables.html new file mode 100644 index 0000000..bea5560 --- /dev/null +++ b/site/docs/2.3.0/api/R/tables.html @@ -0,0 +1,67 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Tables</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for tables {SparkR}"><tr><td>tables {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Tables</h2> + +<h3>Description</h3> + +<p>Returns a SparkDataFrame containing names of tables in the given database. +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +tables(databaseName = NULL) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>databaseName</code></td> +<td> +<p>(optional) name of the database</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>a SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>tables since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p><a href="listTables.html">listTables</a> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D tables("hive") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/take.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/take.html b/site/docs/2.3.0/api/R/take.html new file mode 100644 index 0000000..b244d7c --- /dev/null +++ b/site/docs/2.3.0/api/R/take.html @@ -0,0 +1,110 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Take the first NUM rows of a SparkDataFrame and return the...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for take {SparkR}"><tr><td>take {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Take the first NUM rows of a SparkDataFrame and return the results as a R data.frame</h2> + +<h3>Description</h3> + +<p>Take the first NUM rows of a SparkDataFrame and return the results as a R data.frame +</p> + + +<h3>Usage</h3> + +<pre> +take(x, num) + +## S4 method for signature 'SparkDataFrame,numeric' +take(x, num) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>num</code></td> +<td> +<p>number of rows to take.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>take since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D take(df, 2) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/toJSON.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/toJSON.html b/site/docs/2.3.0/api/R/toJSON.html new file mode 100644 index 0000000..a0e316e --- /dev/null +++ b/site/docs/2.3.0/api/R/toJSON.html @@ -0,0 +1,117 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: toJSON</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for toJSON {SparkR}"><tr><td>toJSON {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>toJSON</h2> + +<h3>Description</h3> + +<p>Converts a SparkDataFrame into a SparkDataFrame of JSON string. +</p> + + +<h3>Usage</h3> + +<pre> +## S4 method for signature 'SparkDataFrame' +toJSON(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Each row is turned into a JSON document with columns as different fields. +The returned SparkDataFrame has a single character column with the name <code>value</code> +</p> + + +<h3>Value</h3> + +<p>a SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>toJSON since 2.2.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.parquet" +##D df <- read.parquet(path) +##D df_json <- toJSON(df) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/uncacheTable.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/uncacheTable.html b/site/docs/2.3.0/api/R/uncacheTable.html new file mode 100644 index 0000000..a4fd9fc --- /dev/null +++ b/site/docs/2.3.0/api/R/uncacheTable.html @@ -0,0 +1,65 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Uncache Table</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for uncacheTable {SparkR}"><tr><td>uncacheTable {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Uncache Table</h2> + +<h3>Description</h3> + +<p>Removes the specified table from the in-memory cache. +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +uncacheTable(tableName) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>tableName</code></td> +<td> +<p>the qualified or unqualified name that designates a table. If no database +identifier is provided, it refers to a table in the current database.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>uncacheTable since 1.4.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D createOrReplaceTempView(df, "table") +##D uncacheTable("table") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/union.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/union.html b/site/docs/2.3.0/api/R/union.html new file mode 100644 index 0000000..d526e3d --- /dev/null +++ b/site/docs/2.3.0/api/R/union.html @@ -0,0 +1,137 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Return a new SparkDataFrame containing the union of rows</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for union {SparkR}"><tr><td>union {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Return a new SparkDataFrame containing the union of rows</h2> + +<h3>Description</h3> + +<p>Return a new SparkDataFrame containing the union of rows in this SparkDataFrame +and another SparkDataFrame. This is equivalent to <code>UNION ALL</code> in SQL. +Input SparkDataFrames can have different schemas (names and data types). +</p> +<p>unionAll is deprecated - use union instead +</p> + + +<h3>Usage</h3> + +<pre> +union(x, y) + +unionAll(x, y) + +## S4 method for signature 'SparkDataFrame,SparkDataFrame' +union(x, y) + +## S4 method for signature 'SparkDataFrame,SparkDataFrame' +unionAll(x, y) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>y</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Note: This does not remove duplicate rows across the two SparkDataFrames. +Also as standard in SQL, this function resolves columns by position (not by name). +</p> + + +<h3>Value</h3> + +<p>A SparkDataFrame containing the result of the union. +</p> + + +<h3>Note</h3> + +<p>union since 2.0.0 +</p> +<p>unionAll since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p><a href="rbind.html">rbind</a> <a href="unionByName.html">unionByName</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="unionByName.html">unionByName</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D df1 <- read.json(path) +##D df2 <- read.json(path2) +##D unioned <- union(df, df2) +##D unions <- rbind(df, df2, df3, df4) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/unionByName.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/unionByName.html b/site/docs/2.3.0/api/R/unionByName.html new file mode 100644 index 0000000..916fe62 --- /dev/null +++ b/site/docs/2.3.0/api/R/unionByName.html @@ -0,0 +1,128 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Return a new SparkDataFrame containing the union of rows,...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for unionByName {SparkR}"><tr><td>unionByName {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Return a new SparkDataFrame containing the union of rows, matched by column names</h2> + +<h3>Description</h3> + +<p>Return a new SparkDataFrame containing the union of rows in this SparkDataFrame +and another SparkDataFrame. This is different from <code>union</code> function, and both +<code>UNION ALL</code> and <code>UNION DISTINCT</code> in SQL as column positions are not taken +into account. Input SparkDataFrames can have different data types in the schema. +</p> + + +<h3>Usage</h3> + +<pre> +unionByName(x, y) + +## S4 method for signature 'SparkDataFrame,SparkDataFrame' +unionByName(x, y) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>y</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Note: This does not remove duplicate rows across the two SparkDataFrames. +This function resolves columns by name (not by position). +</p> + + +<h3>Value</h3> + +<p>A SparkDataFrame containing the result of the union. +</p> + + +<h3>Note</h3> + +<p>unionByName since 2.3.0 +</p> + + +<h3>See Also</h3> + +<p><a href="rbind.html">rbind</a> <a href="union.html">union</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D df1 <- select(createDataFrame(mtcars), "carb", "am", "gear") +##D df2 <- select(createDataFrame(mtcars), "am", "gear", "carb") +##D head(unionByName(df1, df2)) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/unpersist.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/unpersist.html b/site/docs/2.3.0/api/R/unpersist.html new file mode 100644 index 0000000..77d2bc9 --- /dev/null +++ b/site/docs/2.3.0/api/R/unpersist.html @@ -0,0 +1,116 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Unpersist</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for unpersist {SparkR}"><tr><td>unpersist {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Unpersist</h2> + +<h3>Description</h3> + +<p>Mark this SparkDataFrame as non-persistent, and remove all blocks for it from memory and +disk. +</p> + + +<h3>Usage</h3> + +<pre> +unpersist(x, ...) + +## S4 method for signature 'SparkDataFrame' +unpersist(x, blocking = TRUE) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>the SparkDataFrame to unpersist.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>further arguments to be passed to or from other methods.</p> +</td></tr> +<tr valign="top"><td><code>blocking</code></td> +<td> +<p>whether to block until all blocks are deleted.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>unpersist since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="unionByName.html">unionByName</a></code>, <code><a href="union.html">union</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D persist(df, "MEMORY_AND_DISK") +##D unpersist(df) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/windowOrderBy.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/windowOrderBy.html b/site/docs/2.3.0/api/R/windowOrderBy.html new file mode 100644 index 0000000..9bfdfb6 --- /dev/null +++ b/site/docs/2.3.0/api/R/windowOrderBy.html @@ -0,0 +1,71 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: windowOrderBy</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for windowOrderBy {SparkR}"><tr><td>windowOrderBy {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>windowOrderBy</h2> + +<h3>Description</h3> + +<p>Creates a WindowSpec with the ordering defined. +</p> + + +<h3>Usage</h3> + +<pre> +windowOrderBy(col, ...) + +## S4 method for signature 'character' +windowOrderBy(col, ...) + +## S4 method for signature 'Column' +windowOrderBy(col, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>col</code></td> +<td> +<p>A column name or Column by which rows are ordered within +windows.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>Optional column names or Columns in addition to col, by +which rows are ordered within windows.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>windowOrderBy(character) since 2.0.0 +</p> +<p>windowOrderBy(Column) since 2.0.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D ws <- windowOrderBy("key1", "key2") +##D df1 <- select(df, over(lead("value", 1), ws)) +##D +##D ws <- windowOrderBy(df$key1, df$key2) +##D df1 <- select(df, over(lead("value", 1), ws)) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/windowPartitionBy.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/windowPartitionBy.html b/site/docs/2.3.0/api/R/windowPartitionBy.html new file mode 100644 index 0000000..dd55e61 --- /dev/null +++ b/site/docs/2.3.0/api/R/windowPartitionBy.html @@ -0,0 +1,71 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: windowPartitionBy</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for windowPartitionBy {SparkR}"><tr><td>windowPartitionBy {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>windowPartitionBy</h2> + +<h3>Description</h3> + +<p>Creates a WindowSpec with the partitioning defined. +</p> + + +<h3>Usage</h3> + +<pre> +windowPartitionBy(col, ...) + +## S4 method for signature 'character' +windowPartitionBy(col, ...) + +## S4 method for signature 'Column' +windowPartitionBy(col, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>col</code></td> +<td> +<p>A column name or Column by which rows are partitioned to +windows.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>Optional column names or Columns in addition to col, by +which rows are partitioned to windows.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>windowPartitionBy(character) since 2.0.0 +</p> +<p>windowPartitionBy(Column) since 2.0.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D ws <- orderBy(windowPartitionBy("key1", "key2"), "key3") +##D df1 <- select(df, over(lead("value", 1), ws)) +##D +##D ws <- orderBy(windowPartitionBy(df$key1, df$key2), df$key3) +##D df1 <- select(df, over(lead("value", 1), ws)) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/with.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/with.html b/site/docs/2.3.0/api/R/with.html new file mode 100644 index 0000000..1edde70 --- /dev/null +++ b/site/docs/2.3.0/api/R/with.html @@ -0,0 +1,117 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Evaluate a R expression in an environment constructed from a...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for with {SparkR}"><tr><td>with {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Evaluate a R expression in an environment constructed from a SparkDataFrame</h2> + +<h3>Description</h3> + +<p>Evaluate a R expression in an environment constructed from a SparkDataFrame +with() allows access to columns of a SparkDataFrame by simply referring to +their name. It appends every column of a SparkDataFrame into a new +environment. Then, the given expression is evaluated in this new +environment. +</p> + + +<h3>Usage</h3> + +<pre> +with(data, expr, ...) + +## S4 method for signature 'SparkDataFrame' +with(data, expr, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>data</code></td> +<td> +<p>(SparkDataFrame) SparkDataFrame to use for constructing an environment.</p> +</td></tr> +<tr valign="top"><td><code>expr</code></td> +<td> +<p>(expression) Expression to evaluate.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>arguments to be passed to future methods.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>with since 1.6.0 +</p> + + +<h3>See Also</h3> + +<p><a href="attach.html">attach</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="unionByName.html">unionByName</a></code>, <code><a href="union.html">union</a></code>, +<code><a href="unpersist.html">unpersist</a></code>, <code><a href="withColumn.html">withColumn</a></code>, +<code><a href="withWatermark.html">withWatermark</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D with(irisDf, nrow(Sepal_Width)) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/withColumn.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/withColumn.html b/site/docs/2.3.0/api/R/withColumn.html new file mode 100644 index 0000000..d618ae9 --- /dev/null +++ b/site/docs/2.3.0/api/R/withColumn.html @@ -0,0 +1,131 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: WithColumn</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for withColumn {SparkR}"><tr><td>withColumn {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>WithColumn</h2> + +<h3>Description</h3> + +<p>Return a new SparkDataFrame by adding a column or replacing the existing column +that has the same name. +</p> + + +<h3>Usage</h3> + +<pre> +withColumn(x, colName, col) + +## S4 method for signature 'SparkDataFrame,character' +withColumn(x, colName, col) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>colName</code></td> +<td> +<p>a column name.</p> +</td></tr> +<tr valign="top"><td><code>col</code></td> +<td> +<p>a Column expression (which must refer only to this SparkDataFrame), or an atomic +vector in the length of 1 as literal value.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A SparkDataFrame with the new column added or the existing column replaced. +</p> + + +<h3>Note</h3> + +<p>withColumn since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p><a href="rename.html">rename</a> <a href="mutate.html">mutate</a> <a href="subset.html">subset</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="unionByName.html">unionByName</a></code>, <code><a href="union.html">union</a></code>, +<code><a href="unpersist.html">unpersist</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D newDF <- withColumn(df, "newCol", df$col1 * 5) +##D # Replace an existing column +##D newDF2 <- withColumn(newDF, "newCol", newDF$col1) +##D newDF3 <- withColumn(newDF, "newCol", 42) +##D # Use extract operator to set an existing or new column +##D df[["age"]] <- 23 +##D df[[2]] <- df$col1 +##D df[[2]] <- NULL # drop column +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/withWatermark.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/withWatermark.html b/site/docs/2.3.0/api/R/withWatermark.html new file mode 100644 index 0000000..b0894d9 --- /dev/null +++ b/site/docs/2.3.0/api/R/withWatermark.html @@ -0,0 +1,147 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: withWatermark</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for withWatermark {SparkR}"><tr><td>withWatermark {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>withWatermark</h2> + +<h3>Description</h3> + +<p>Defines an event time watermark for this streaming SparkDataFrame. A watermark tracks a point in +time before which we assume no more late data is going to arrive. +</p> + + +<h3>Usage</h3> + +<pre> +withWatermark(x, eventTime, delayThreshold) + +## S4 method for signature 'SparkDataFrame,character,character' +withWatermark(x, eventTime, + delayThreshold) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a streaming SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>eventTime</code></td> +<td> +<p>a string specifying the name of the Column that contains the event time of the +row.</p> +</td></tr> +<tr valign="top"><td><code>delayThreshold</code></td> +<td> +<p>a string specifying the minimum delay to wait to data to arrive late, +relative to the latest record that has been processed in the form of an +interval (e.g. "1 minute" or "5 hours"). NOTE: This should not be negative.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Spark will use this watermark for several purposes: +</p> + +<ul> +<li><p> To know when a given time window aggregation can be finalized and thus can be emitted +when using output modes that do not allow updates. +</p> +</li> +<li><p> To minimize the amount of state that we need to keep for on-going aggregations. +</p> +</li></ul> + +<p>The current watermark is computed by looking at the <code>MAX(eventTime)</code> seen across +all of the partitions in the query minus a user specified <code>delayThreshold</code>. Due to the cost +of coordinating this value across partitions, the actual watermark used is only guaranteed +to be at least <code>delayThreshold</code> behind the actual event time. In some cases we may still +process records that arrive more than <code>delayThreshold</code> late. +</p> + + +<h3>Value</h3> + +<p>a SparkDataFrame. +</p> + + +<h3>Note</h3> + +<p>withWatermark since 2.3.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="unionByName.html">unionByName</a></code>, <code><a href="union.html">union</a></code>, +<code><a href="unpersist.html">unpersist</a></code>, <code><a href="withColumn.html">withColumn</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D schema <- structType(structField("time", "timestamp"), structField("value", "double")) +##D df <- read.stream("json", path = jsonDir, schema = schema, maxFilesPerTrigger = 1) +##D df <- withWatermark(df, "time", "10 minutes") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/26c57a24/site/docs/2.3.0/api/R/write.df.html ---------------------------------------------------------------------- diff --git a/site/docs/2.3.0/api/R/write.df.html b/site/docs/2.3.0/api/R/write.df.html new file mode 100644 index 0000000..453a5ce --- /dev/null +++ b/site/docs/2.3.0/api/R/write.df.html @@ -0,0 +1,161 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Save the contents of SparkDataFrame to a data source.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for write.df {SparkR}"><tr><td>write.df {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Save the contents of SparkDataFrame to a data source.</h2> + +<h3>Description</h3> + +<p>The data source is specified by the <code>source</code> and a set of options (...). +If <code>source</code> is not specified, the default data source configured by +spark.sql.sources.default will be used. +</p> + + +<h3>Usage</h3> + +<pre> +write.df(df, path = NULL, ...) + +saveDF(df, path, source = NULL, mode = "error", ...) + +write.df(df, path = NULL, ...) + +## S4 method for signature 'SparkDataFrame' +write.df(df, path = NULL, source = NULL, + mode = "error", ...) + +## S4 method for signature 'SparkDataFrame,character' +saveDF(df, path, source = NULL, + mode = "error", ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>df</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>path</code></td> +<td> +<p>a name for the table.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional argument(s) passed to the method.</p> +</td></tr> +<tr valign="top"><td><code>source</code></td> +<td> +<p>a name for external data source.</p> +</td></tr> +<tr valign="top"><td><code>mode</code></td> +<td> +<p>one of 'append', 'overwrite', 'error', 'errorifexists', 'ignore' +save mode (it is 'error' by default)</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Additionally, mode is used to specify the behavior of the save operation when data already +exists in the data source. There are four modes: +</p> + +<ul> +<li><p> 'append': Contents of this SparkDataFrame are expected to be appended to existing data. +</p> +</li> +<li><p> 'overwrite': Existing data is expected to be overwritten by the contents of this +SparkDataFrame. +</p> +</li> +<li><p> 'error' or 'errorifexists': An exception is expected to be thrown. +</p> +</li> +<li><p> 'ignore': The save operation is expected to not save the contents of the SparkDataFrame +and to not change the existing data. +</p> +</li></ul> + + + +<h3>Note</h3> + +<p>write.df since 1.4.0 +</p> +<p>saveDF since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="rbind.html">rbind</a></code>, <code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, +<code><a href="rename.html">rename</a></code>, <code><a href="repartition.html">repartition</a></code>, +<code><a href="rollup.html">rollup</a></code>, <code><a href="sample.html">sample</a></code>, +<code><a href="saveAsTable.html">saveAsTable</a></code>, <code><a href="schema.html">schema</a></code>, +<code><a href="selectExpr.html">selectExpr</a></code>, <code><a href="select.html">select</a></code>, +<code><a href="showDF.html">showDF</a></code>, <code><a href="show.html">show</a></code>, +<code><a href="storageLevel.html">storageLevel</a></code>, <code><a href="str.html">str</a></code>, +<code><a href="subset.html">subset</a></code>, <code><a href="summary.html">summary</a></code>, +<code><a href="take.html">take</a></code>, <code><a href="toJSON.html">toJSON</a></code>, +<code><a href="unionByName.html">unionByName</a></code>, <code><a href="union.html">union</a></code>, +<code><a href="unpersist.html">unpersist</a></code>, <code><a href="withColumn.html">withColumn</a></code>, +<code><a href="withWatermark.html">withWatermark</a></code>, <code><a href="with.html">with</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D write.df(df, "myfile", "parquet", "overwrite") +##D saveDF(df, parquetPath2, "parquet", mode = "append", mergeSchema = TRUE) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org