Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/mutate.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/mutate.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/mutate.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,135 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Mutate</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for mutate {SparkR}"><tr><td>mutate {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Mutate</h2> + +<h3>Description</h3> + +<p>Return a new SparkDataFrame with the specified columns added or replaced. +</p> + + +<h3>Usage</h3> + +<pre> +mutate(.data, ...) + +transform(`_data`, ...) + +## S4 method for signature 'SparkDataFrame' +mutate(.data, ...) + +## S4 method for signature 'SparkDataFrame' +transform(`_data`, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>.data</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional column argument(s) each in the form name = col.</p> +</td></tr> +<tr valign="top"><td><code>_data</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A new SparkDataFrame with the new columns added or replaced. +</p> + + +<h3>Note</h3> + +<p>mutate since 1.4.0 +</p> +<p>transform since 1.5.0 +</p> + + +<h3>See Also</h3> + +<p><a href="rename.html">rename</a> <a href="withColumn.html">withColumn</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="ncol.html">ncol</a></code>, <code><a href="nrow.html">nrow</a></code>, +<code><a href="persist.html">persist</a></code>, <code><a href="printSchema.html">printSchema</a></code>, +<code><a href="randomSplit.html">randomSplit</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D newDF <- mutate(df, newCol = df$col1 * 5, newCol2 = df$col1 * 2) +##D names(newDF) # Will contain newCol, newCol2 +##D newDF2 <- transform(df, newCol = df$col1 / 5, newCol2 = df$col1 * 2) +##D +##D df <- createDataFrame(list(list("Andy", 30L), list("Justin", 19L)), c("name", "age")) +##D # Replace the "age" column +##D df1 <- mutate(df, age = df$age + 1L) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html>
Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/nafunctions.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/nafunctions.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/nafunctions.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,175 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: A set of SparkDataFrame functions working with NA values</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for dropna {SparkR}"><tr><td>dropna {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>A set of SparkDataFrame functions working with NA values</h2> + +<h3>Description</h3> + +<p>dropna, na.omit - Returns a new SparkDataFrame omitting rows with null values. +</p> +<p>fillna - Replace null values. +</p> + + +<h3>Usage</h3> + +<pre> +dropna(x, how = c("any", "all"), minNonNulls = NULL, cols = NULL) + +na.omit(object, ...) + +fillna(x, value, cols = NULL) + +## S4 method for signature 'SparkDataFrame' +dropna(x, how = c("any", "all"), + minNonNulls = NULL, cols = NULL) + +## S4 method for signature 'SparkDataFrame' +na.omit(object, how = c("any", "all"), + minNonNulls = NULL, cols = NULL) + +## S4 method for signature 'SparkDataFrame' +fillna(x, value, cols = NULL) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>how</code></td> +<td> +<p>"any" or "all". +if "any", drop a row if it contains any nulls. +if "all", drop a row only if all its values are null. +if <code>minNonNulls</code> is specified, how is ignored.</p> +</td></tr> +<tr valign="top"><td><code>minNonNulls</code></td> +<td> +<p>if specified, drop rows that have less than +<code>minNonNulls</code> non-null values. +This overwrites the how parameter.</p> +</td></tr> +<tr valign="top"><td><code>cols</code></td> +<td> +<p>optional list of column names to consider. In <code>fillna</code>, +columns specified in cols that do not have matching data +type are ignored. For example, if value is a character, and +subset contains a non-character column, then the non-character +column is simply ignored.</p> +</td></tr> +<tr valign="top"><td><code>object</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>further arguments to be passed to or from other methods.</p> +</td></tr> +<tr valign="top"><td><code>value</code></td> +<td> +<p>value to replace null values with. +Should be an integer, numeric, character or named list. +If the value is a named list, then cols is ignored and +value must be a mapping from column name (character) to +replacement value. The replacement value must be an +integer, numeric or character.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A SparkDataFrame. +</p> + + +<h3>Note</h3> + +<p>dropna since 1.4.0 +</p> +<p>na.omit since 1.5.0 +</p> +<p>fillna since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="drop.html">drop</a></code>, <code><a href="dtypes.html">dtypes</a></code>, +<code><a href="except.html">except</a></code>, <code><a href="explain.html">explain</a></code>, +<code><a href="filter.html">filter</a></code>, <code><a href="first.html">first</a></code>, +<code><a href="gapplyCollect.html">gapplyCollect</a></code>, <code><a href="gapply.html">gapply</a></code>, +<code><a href="getNumPartitions.html">getNumPartitions</a></code>, <code><a href="groupBy.html">group_by</a></code>, +<code><a href="head.html">head</a></code>, <code><a href="hint.html">hint</a></code>, +<code><a href="histogram.html">histogram</a></code>, <code><a href="insertInto.html">insertInto</a></code>, +<code><a href="intersect.html">intersect</a></code>, <code><a href="isLocal.html">isLocal</a></code>, +<code><a href="isStreaming.html">isStreaming</a></code>, <code><a href="join.html">join</a></code>, +<code><a href="limit.html">limit</a></code>, <code><a href="localCheckpoint.html">localCheckpoint</a></code>, +<code><a href="merge.html">merge</a></code>, <code><a href="mutate.html">mutate</a></code>, +<code><a href="ncol.html">ncol</a></code>, <code><a href="nrow.html">nrow</a></code>, +<code><a href="persist.html">persist</a></code>, <code><a href="printSchema.html">printSchema</a></code>, +<code><a href="randomSplit.html">randomSplit</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D dropna(df) +## End(Not run) +## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D fillna(df, 1) +##D fillna(df, list("age" = 20, "name" = "unknown")) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/ncol.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/ncol.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/ncol.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,104 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Returns the number of columns in a SparkDataFrame</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for ncol {SparkR}"><tr><td>ncol {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Returns the number of columns in a SparkDataFrame</h2> + +<h3>Description</h3> + +<p>Returns the number of columns in a SparkDataFrame +</p> + + +<h3>Usage</h3> + +<pre> +## S4 method for signature 'SparkDataFrame' +ncol(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>ncol since 1.5.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="nrow.html">nrow</a></code>, +<code><a href="persist.html">persist</a></code>, <code><a href="printSchema.html">printSchema</a></code>, +<code><a href="randomSplit.html">randomSplit</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D ncol(df) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/not.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/not.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/not.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,90 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: !</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for not {SparkR}"><tr><td>not {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>!</h2> + +<h3>Description</h3> + +<p>Inversion of boolean expression. +</p> +<p>Inversion of boolean expression. +</p> + + +<h3>Usage</h3> + +<pre> +not(x) + +## S4 method for signature 'Column' +!x + +## S4 method for signature 'Column' +not(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>Column to compute on</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p><code>not</code> and <code>!</code> cannot be applied directly to numerical column. +To achieve R-like truthiness column has to be casted to <code>BooleanType</code>. +</p> + + +<h3>Note</h3> + +<p>! since 2.3.0 +</p> +<p>not since 2.3.0 +</p> + + +<h3>See Also</h3> + +<p>Other non-aggregate functions: <code><a href="column_nonaggregate_functions.html">column_nonaggregate_functions</a></code>, +<code><a href="column.html">column</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D df <- createDataFrame(data.frame(x = c(-1, 0, 1))) +##D +##D head(select(df, !column("x") > 0)) +## End(Not run) +## Not run: +##D df <- createDataFrame(data.frame( +##D is_true = c(TRUE, FALSE, NA), +##D flag = c(1, 0, 1) +##D )) +##D +##D head(select(df, not(df$is_true))) +##D +##D # Explicit cast is required when working with numeric column +##D head(select(df, not(cast(df$flag, "boolean")))) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/nrow.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/nrow.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/nrow.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,109 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Returns the number of rows in a SparkDataFrame</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for nrow {SparkR}"><tr><td>nrow {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Returns the number of rows in a SparkDataFrame</h2> + +<h3>Description</h3> + +<p>Returns the number of rows in a SparkDataFrame +</p> + + +<h3>Usage</h3> + +<pre> +## S4 method for signature 'SparkDataFrame' +count(x) + +## S4 method for signature 'SparkDataFrame' +nrow(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>count since 1.4.0 +</p> +<p>nrow since 1.5.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="persist.html">persist</a></code>, <code><a href="printSchema.html">printSchema</a></code>, +<code><a href="randomSplit.html">randomSplit</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D count(df) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/orderBy.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/orderBy.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/orderBy.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,85 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Ordering Columns in a WindowSpec</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for orderBy {SparkR}"><tr><td>orderBy {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Ordering Columns in a WindowSpec</h2> + +<h3>Description</h3> + +<p>Defines the ordering columns in a WindowSpec. +</p> + + +<h3>Usage</h3> + +<pre> +orderBy(x, col, ...) + +## S4 method for signature 'WindowSpec,character' +orderBy(x, col, ...) + +## S4 method for signature 'WindowSpec,Column' +orderBy(x, col, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a WindowSpec</p> +</td></tr> +<tr valign="top"><td><code>col</code></td> +<td> +<p>a character or Column indicating an ordering column</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional sorting fields</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A WindowSpec. +</p> + + +<h3>Note</h3> + +<p>orderBy(WindowSpec, character) since 2.0.0 +</p> +<p>orderBy(WindowSpec, Column) since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p>See <a href="arrange.html">arrange</a> for use in sorting a SparkDataFrame +</p> +<p>Other windowspec_method: <code><a href="partitionBy.html">partitionBy</a></code>, +<code><a href="rangeBetween.html">rangeBetween</a></code>, <code><a href="rowsBetween.html">rowsBetween</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D orderBy(ws, "col1", "col2") +##D orderBy(ws, df$col1, df$col2) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/otherwise.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/otherwise.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/otherwise.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,57 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: otherwise</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for otherwise {SparkR}"><tr><td>otherwise {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>otherwise</h2> + +<h3>Description</h3> + +<p>If values in the specified column are null, returns the value. +Can be used in conjunction with <code>when</code> to specify a default value for expressions. +</p> + + +<h3>Usage</h3> + +<pre> +otherwise(x, value) + +## S4 method for signature 'Column' +otherwise(x, value) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a Column.</p> +</td></tr> +<tr valign="top"><td><code>value</code></td> +<td> +<p>value to replace when the corresponding entry in <code>x</code> is NA. +Can be a single value or a Column.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>otherwise since 1.5.0 +</p> + + +<h3>See Also</h3> + +<p>Other colum_func: <code><a href="alias.html">alias</a></code>, +<code><a href="between.html">between</a></code>, <code><a href="cast.html">cast</a></code>, +<code><a href="endsWith.html">endsWith</a></code>, <code><a href="over.html">over</a></code>, +<code><a href="startsWith.html">startsWith</a></code>, <code><a href="substr.html">substr</a></code> +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/over.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/over.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/over.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,79 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: over</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for over {SparkR}"><tr><td>over {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>over</h2> + +<h3>Description</h3> + +<p>Define a windowing column. +</p> + + +<h3>Usage</h3> + +<pre> +over(x, window) + +## S4 method for signature 'Column,WindowSpec' +over(x, window) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a Column, usually one returned by window function(s).</p> +</td></tr> +<tr valign="top"><td><code>window</code></td> +<td> +<p>a WindowSpec object. Can be created by <code>windowPartitionBy</code> or +<code>windowOrderBy</code> and configured by other WindowSpec methods.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>over since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p>Other colum_func: <code><a href="alias.html">alias</a></code>, +<code><a href="between.html">between</a></code>, <code><a href="cast.html">cast</a></code>, +<code><a href="endsWith.html">endsWith</a></code>, <code><a href="otherwise.html">otherwise</a></code>, +<code><a href="startsWith.html">startsWith</a></code>, <code><a href="substr.html">substr</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D df <- createDataFrame(mtcars) +##D +##D # Partition by am (transmission) and order by hp (horsepower) +##D ws <- orderBy(windowPartitionBy("am"), "hp") +##D +##D # Rank on hp within each partition +##D out <- select(df, over(rank(), ws), df$hp, df$am) +##D +##D # Lag mpg values by 1 row on the partition-and-ordered table +##D out <- select(df, over(lead(df$mpg), ws), df$mpg, df$hp, df$am) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/partitionBy.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/partitionBy.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/partitionBy.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,78 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: partitionBy</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for partitionBy {SparkR}"><tr><td>partitionBy {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>partitionBy</h2> + +<h3>Description</h3> + +<p>Defines the partitioning columns in a WindowSpec. +</p> + + +<h3>Usage</h3> + +<pre> +partitionBy(x, ...) + +## S4 method for signature 'WindowSpec' +partitionBy(x, col, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a WindowSpec.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional column(s) to partition on.</p> +</td></tr> +<tr valign="top"><td><code>col</code></td> +<td> +<p>a column to partition on (desribed by the name or Column).</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>A WindowSpec. +</p> + + +<h3>Note</h3> + +<p>partitionBy(WindowSpec) since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p>Other windowspec_method: <code><a href="orderBy.html">orderBy</a></code>, +<code><a href="rangeBetween.html">rangeBetween</a></code>, <code><a href="rowsBetween.html">rowsBetween</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D partitionBy(ws, "col1", "col2") +##D partitionBy(ws, df$col1, df$col2) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/persist.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/persist.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/persist.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,113 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Persist</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for persist {SparkR}"><tr><td>persist {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Persist</h2> + +<h3>Description</h3> + +<p>Persist this SparkDataFrame with the specified storage level. For details of the +supported storage levels, refer to +<a href="http://spark.apache.org/docs/latest/rdd-programming-guide.html#rdd-persistence">http://spark.apache.org/docs/latest/rdd-programming-guide.html#rdd-persistence</a>. +</p> + + +<h3>Usage</h3> + +<pre> +persist(x, newLevel) + +## S4 method for signature 'SparkDataFrame,character' +persist(x, newLevel) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>the SparkDataFrame to persist.</p> +</td></tr> +<tr valign="top"><td><code>newLevel</code></td> +<td> +<p>storage level chosen for the persistance. See available options in +the description.</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>persist since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="printSchema.html">printSchema</a></code>, +<code><a href="randomSplit.html">randomSplit</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D persist(df, "MEMORY_AND_DISK") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/pivot.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/pivot.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/pivot.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,80 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Pivot a column of the GroupedData and perform the specified...</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for pivot {SparkR}"><tr><td>pivot {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Pivot a column of the GroupedData and perform the specified aggregation.</h2> + +<h3>Description</h3> + +<p>Pivot a column of the GroupedData and perform the specified aggregation. +There are two versions of pivot function: one that requires the caller to specify the list +of distinct values to pivot on, and one that does not. The latter is more concise but less +efficient, because Spark needs to first compute the list of distinct values internally. +</p> + + +<h3>Usage</h3> + +<pre> +## S4 method for signature 'GroupedData,character' +pivot(x, colname, values = list()) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a GroupedData object</p> +</td></tr> +<tr valign="top"><td><code>colname</code></td> +<td> +<p>A column name</p> +</td></tr> +<tr valign="top"><td><code>values</code></td> +<td> +<p>A value or a list/vector of distinct values for the output columns.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>GroupedData object +</p> + + +<h3>Note</h3> + +<p>pivot since 2.0.0 +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D df <- createDataFrame(data.frame( +##D earnings = c(10000, 10000, 11000, 15000, 12000, 20000, 21000, 22000), +##D course = c("R", "Python", "R", "Python", "R", "Python", "R", "Python"), +##D period = c("1H", "1H", "2H", "2H", "1H", "1H", "2H", "2H"), +##D year = c(2015, 2015, 2015, 2015, 2016, 2016, 2016, 2016) +##D )) +##D group_sum <- sum(pivot(groupBy(df, "year"), "course"), "earnings") +##D group_min <- min(pivot(groupBy(df, "year"), "course", "R"), "earnings") +##D group_max <- max(pivot(groupBy(df, "year"), "course", c("Python", "R")), "earnings") +##D group_mean <- mean(pivot(groupBy(df, "year"), "course", list("Python", "R")), "earnings") +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/predict.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/predict.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/predict.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,54 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Makes predictions from a MLlib model</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for predict {SparkR}"><tr><td>predict {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Makes predictions from a MLlib model</h2> + +<h3>Description</h3> + +<p>Makes predictions from a MLlib model. For more information, see the specific +MLlib model below. +</p> + + +<h3>Usage</h3> + +<pre> +predict(object, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>object</code></td> +<td> +<p>a fitted ML model object.</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional argument(s) passed to the method.</p> +</td></tr> +</table> + + +<h3>See Also</h3> + +<p><a href="spark.als.html">spark.als</a>, <a href="spark.bisectingKmeans.html">spark.bisectingKmeans</a>, <a href="spark.decisionTree.html">spark.decisionTree</a>, +</p> +<p><a href="spark.gaussianMixture.html">spark.gaussianMixture</a>, <a href="spark.gbt.html">spark.gbt</a>, +</p> +<p><a href="spark.glm.html">spark.glm</a>, <a href="glm.html">glm</a>, <a href="spark.isoreg.html">spark.isoreg</a>, +</p> +<p><a href="spark.kmeans.html">spark.kmeans</a>, +</p> +<p><a href="spark.logit.html">spark.logit</a>, <a href="spark.mlp.html">spark.mlp</a>, <a href="spark.naiveBayes.html">spark.naiveBayes</a>, +</p> +<p><a href="spark.randomForest.html">spark.randomForest</a>, <a href="spark.survreg.html">spark.survreg</a>, <a href="spark.svmLinear.html">spark.svmLinear</a> +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/print.jobj.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/print.jobj.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/print.jobj.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,45 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Print a JVM object reference.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for print.jobj {SparkR}"><tr><td>print.jobj {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Print a JVM object reference.</h2> + +<h3>Description</h3> + +<p>This function prints the type and id for an object stored +in the SparkR JVM backend. +</p> + + +<h3>Usage</h3> + +<pre> +## S3 method for class 'jobj' +print(x, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>The JVM object reference</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>further arguments passed to or from other methods</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>print.jobj since 1.4.0 +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/print.structField.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/print.structField.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/print.structField.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,45 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Print a Spark StructField.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for print.structField {SparkR}"><tr><td>print.structField {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Print a Spark StructField.</h2> + +<h3>Description</h3> + +<p>This function prints the contents of a StructField returned from the +SparkR JVM backend. +</p> + + +<h3>Usage</h3> + +<pre> +## S3 method for class 'structField' +print(x, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A StructField object</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>further arguments passed to or from other methods</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>print.structField since 1.4.0 +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/print.structType.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/print.structType.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/print.structType.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,45 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Print a Spark StructType.</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> +</head><body> + +<table width="100%" summary="page for print.structType {SparkR}"><tr><td>print.structType {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Print a Spark StructType.</h2> + +<h3>Description</h3> + +<p>This function prints the contents of a StructType returned from the +SparkR JVM backend. +</p> + + +<h3>Usage</h3> + +<pre> +## S3 method for class 'structType' +print(x, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A StructType object</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>further arguments passed to or from other methods</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>print.structType since 1.4.0 +</p> + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/printSchema.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/printSchema.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/printSchema.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,106 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Print Schema of a SparkDataFrame</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for printSchema {SparkR}"><tr><td>printSchema {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Print Schema of a SparkDataFrame</h2> + +<h3>Description</h3> + +<p>Prints out the schema in tree format +</p> + + +<h3>Usage</h3> + +<pre> +printSchema(x) + +## S4 method for signature 'SparkDataFrame' +printSchema(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>printSchema since 1.4.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="randomSplit.html">randomSplit</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D path <- "path/to/file.json" +##D df <- read.json(path) +##D printSchema(df) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/queryName.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/queryName.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/queryName.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,75 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: queryName</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for queryName {SparkR}"><tr><td>queryName {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>queryName</h2> + +<h3>Description</h3> + +<p>Returns the user-specified name of the query. This is specified in +<code>write.stream(df, queryName = "query")</code>. This name, if set, must be unique across all active +queries. +</p> + + +<h3>Usage</h3> + +<pre> +queryName(x) + +## S4 method for signature 'StreamingQuery' +queryName(x) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a StreamingQuery.</p> +</td></tr> +</table> + + +<h3>Value</h3> + +<p>The name of the query, or NULL if not specified. +</p> + + +<h3>Note</h3> + +<p>queryName(StreamingQuery) since 2.2.0 +</p> +<p>experimental +</p> + + +<h3>See Also</h3> + +<p><a href="write.stream.html">write.stream</a> +</p> +<p>Other StreamingQuery methods: <code><a href="awaitTermination.html">awaitTermination</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="isActive.html">isActive</a></code>, +<code><a href="lastProgress.html">lastProgress</a></code>, <code><a href="status.html">status</a></code>, +<code><a href="stopQuery.html">stopQuery</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: queryName(sq) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/randomSplit.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/randomSplit.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/randomSplit.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,115 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: randomSplit</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for randomSplit {SparkR}"><tr><td>randomSplit {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>randomSplit</h2> + +<h3>Description</h3> + +<p>Return a list of randomly split dataframes with the provided weights. +</p> + + +<h3>Usage</h3> + +<pre> +randomSplit(x, weights, seed) + +## S4 method for signature 'SparkDataFrame,numeric' +randomSplit(x, weights, seed) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>A SparkDataFrame</p> +</td></tr> +<tr valign="top"><td><code>weights</code></td> +<td> +<p>A vector of weights for splits, will be normalized if they don't sum to 1</p> +</td></tr> +<tr valign="top"><td><code>seed</code></td> +<td> +<p>A seed to use for random split</p> +</td></tr> +</table> + + +<h3>Note</h3> + +<p>randomSplit since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="rbind.html">rbind</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D df <- createDataFrame(data.frame(id = 1:1000)) +##D df_list <- randomSplit(df, c(2, 3, 5), 0) +##D # df_list contains 3 SparkDataFrames with each having about 200, 300 and 500 rows respectively +##D sapply(df_list, count) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/rangeBetween.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/rangeBetween.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/rangeBetween.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,87 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: rangeBetween</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for rangeBetween {SparkR}"><tr><td>rangeBetween {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>rangeBetween</h2> + +<h3>Description</h3> + +<p>Defines the frame boundaries, from <code>start</code> (inclusive) to <code>end</code> (inclusive). +</p> + + +<h3>Usage</h3> + +<pre> +rangeBetween(x, start, end) + +## S4 method for signature 'WindowSpec,numeric,numeric' +rangeBetween(x, start, end) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a WindowSpec</p> +</td></tr> +<tr valign="top"><td><code>start</code></td> +<td> +<p>boundary start, inclusive. +The frame is unbounded if this is the minimum long value.</p> +</td></tr> +<tr valign="top"><td><code>end</code></td> +<td> +<p>boundary end, inclusive. +The frame is unbounded if this is the maximum long value.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Both <code>start</code> and <code>end</code> are relative from the current row. For example, "0" means +"current row", while "-1" means one off before the current row, and "5" means the five off +after the current row. +</p> + + +<h3>Value</h3> + +<p>a WindowSpec +</p> + + +<h3>Note</h3> + +<p>rangeBetween since 2.0.0 +</p> + + +<h3>See Also</h3> + +<p>Other windowspec_method: <code><a href="orderBy.html">orderBy</a></code>, +<code><a href="partitionBy.html">partitionBy</a></code>, <code><a href="rowsBetween.html">rowsBetween</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D rangeBetween(ws, 0, 3) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/rbind.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/rbind.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/rbind.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,128 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Union two or more SparkDataFrames</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for rbind {SparkR}"><tr><td>rbind {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Union two or more SparkDataFrames</h2> + +<h3>Description</h3> + +<p>Union two or more SparkDataFrames by row. As in R's <code>rbind</code>, this method +requires that the input SparkDataFrames have the same column names. +</p> + + +<h3>Usage</h3> + +<pre> +rbind(..., deparse.level = 1) + +## S4 method for signature 'SparkDataFrame' +rbind(x, ..., deparse.level = 1) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional SparkDataFrame(s).</p> +</td></tr> +<tr valign="top"><td><code>deparse.level</code></td> +<td> +<p>currently not used (put here to match the signature of +the base implementation).</p> +</td></tr> +<tr valign="top"><td><code>x</code></td> +<td> +<p>a SparkDataFrame.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>Note: This does not remove duplicate rows across the two SparkDataFrames. +</p> + + +<h3>Value</h3> + +<p>A SparkDataFrame containing the result of the union. +</p> + + +<h3>Note</h3> + +<p>rbind since 1.5.0 +</p> + + +<h3>See Also</h3> + +<p><a href="union.html">union</a> <a href="unionByName.html">unionByName</a> +</p> +<p>Other SparkDataFrame functions: <code><a href="SparkDataFrame.html">SparkDataFrame-class</a></code>, +<code><a href="summarize.html">agg</a></code>, <code><a href="alias.html">alias</a></code>, +<code><a href="arrange.html">arrange</a></code>, <code><a href="as.data.frame.html">as.data.frame</a></code>, +<code><a href="attach.html">attach,SparkDataFrame-method</a></code>, +<code><a href="broadcast.html">broadcast</a></code>, <code><a href="cache.html">cache</a></code>, +<code><a href="checkpoint.html">checkpoint</a></code>, <code><a href="coalesce.html">coalesce</a></code>, +<code><a href="collect.html">collect</a></code>, <code><a href="columns.html">colnames</a></code>, +<code><a href="coltypes.html">coltypes</a></code>, +<code><a href="createOrReplaceTempView.html">createOrReplaceTempView</a></code>, +<code><a href="crossJoin.html">crossJoin</a></code>, <code><a href="cube.html">cube</a></code>, +<code><a href="dapplyCollect.html">dapplyCollect</a></code>, <code><a href="dapply.html">dapply</a></code>, +<code><a href="describe.html">describe</a></code>, <code><a href="dim.html">dim</a></code>, +<code><a href="distinct.html">distinct</a></code>, <code><a href="dropDuplicates.html">dropDuplicates</a></code>, +<code><a href="nafunctions.html">dropna</a></code>, <code><a href="drop.html">drop</a></code>, +<code><a href="dtypes.html">dtypes</a></code>, <code><a href="except.html">except</a></code>, +<code><a href="explain.html">explain</a></code>, <code><a href="filter.html">filter</a></code>, +<code><a href="first.html">first</a></code>, <code><a href="gapplyCollect.html">gapplyCollect</a></code>, +<code><a href="gapply.html">gapply</a></code>, <code><a href="getNumPartitions.html">getNumPartitions</a></code>, +<code><a href="groupBy.html">group_by</a></code>, <code><a href="head.html">head</a></code>, +<code><a href="hint.html">hint</a></code>, <code><a href="histogram.html">histogram</a></code>, +<code><a href="insertInto.html">insertInto</a></code>, <code><a href="intersect.html">intersect</a></code>, +<code><a href="isLocal.html">isLocal</a></code>, <code><a href="isStreaming.html">isStreaming</a></code>, +<code><a href="join.html">join</a></code>, <code><a href="limit.html">limit</a></code>, +<code><a href="localCheckpoint.html">localCheckpoint</a></code>, <code><a href="merge.html">merge</a></code>, +<code><a href="mutate.html">mutate</a></code>, <code><a href="ncol.html">ncol</a></code>, +<code><a href="nrow.html">nrow</a></code>, <code><a href="persist.html">persist</a></code>, +<code><a href="printSchema.html">printSchema</a></code>, <code><a href="randomSplit.html">randomSplit</a></code>, +<code><a href="registerTempTable-deprecated.html">registerTempTable</a></code>, <code><a href="rename.html">rename</a></code>, +<code><a href="repartition.html">repartition</a></code>, <code><a href="rollup.html">rollup</a></code>, +<code><a href="sample.html">sample</a></code>, <code><a href="saveAsTable.html">saveAsTable</a></code>, +<code><a href="schema.html">schema</a></code>, <code><a href="selectExpr.html">selectExpr</a></code>, +<code><a href="select.html">select</a></code>, <code><a href="showDF.html">showDF</a></code>, +<code><a href="show.html">show</a></code>, <code><a href="storageLevel.html">storageLevel</a></code>, +<code><a href="str.html">str</a></code>, <code><a href="subset.html">subset</a></code>, +<code><a href="summary.html">summary</a></code>, <code><a href="take.html">take</a></code>, +<code><a href="toJSON.html">toJSON</a></code>, <code><a href="unionByName.html">unionByName</a></code>, +<code><a href="union.html">union</a></code>, <code><a href="unpersist.html">unpersist</a></code>, +<code><a href="withColumn.html">withColumn</a></code>, <code><a href="withWatermark.html">withWatermark</a></code>, +<code><a href="with.html">with</a></code>, <code><a href="write.df.html">write.df</a></code>, +<code><a href="write.jdbc.html">write.jdbc</a></code>, <code><a href="write.json.html">write.json</a></code>, +<code><a href="write.orc.html">write.orc</a></code>, <code><a href="write.parquet.html">write.parquet</a></code>, +<code><a href="write.stream.html">write.stream</a></code>, <code><a href="write.text.html">write.text</a></code> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D unions <- rbind(df, df2, df3, df4) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> Added: dev/spark/v2.3.0-rc1-docs/_site/api/R/read.df.html ============================================================================== --- dev/spark/v2.3.0-rc1-docs/_site/api/R/read.df.html (added) +++ dev/spark/v2.3.0-rc1-docs/_site/api/R/read.df.html Sat Jan 13 10:29:47 2018 @@ -0,0 +1,105 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Load a SparkDataFrame</title> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<link rel="stylesheet" type="text/css" href="R.css" /> + +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/styles/github.min.css"> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/highlight.min.js"></script> +<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/8.3/languages/r.min.js"></script> +<script>hljs.initHighlightingOnLoad();</script> +</head><body> + +<table width="100%" summary="page for read.df {SparkR}"><tr><td>read.df {SparkR}</td><td style="text-align: right;">R Documentation</td></tr></table> + +<h2>Load a SparkDataFrame</h2> + +<h3>Description</h3> + +<p>Returns the dataset in a data source as a SparkDataFrame +</p> + + +<h3>Usage</h3> + +<pre> +## Default S3 method: +read.df(path = NULL, source = NULL, schema = NULL, + na.strings = "NA", ...) + +## Default S3 method: +loadDF(path = NULL, source = NULL, schema = NULL, ...) +</pre> + + +<h3>Arguments</h3> + +<table summary="R argblock"> +<tr valign="top"><td><code>path</code></td> +<td> +<p>The path of files to load</p> +</td></tr> +<tr valign="top"><td><code>source</code></td> +<td> +<p>The name of external data source</p> +</td></tr> +<tr valign="top"><td><code>schema</code></td> +<td> +<p>The data schema defined in structType or a DDL-formatted string.</p> +</td></tr> +<tr valign="top"><td><code>na.strings</code></td> +<td> +<p>Default string value for NA when source is "csv"</p> +</td></tr> +<tr valign="top"><td><code>...</code></td> +<td> +<p>additional external data source specific named properties.</p> +</td></tr> +</table> + + +<h3>Details</h3> + +<p>The data source is specified by the <code>source</code> and a set of options(...). +If <code>source</code> is not specified, the default data source configured by +"spark.sql.sources.default" will be used. <br /> +Similar to R read.csv, when <code>source</code> is "csv", by default, a value of "NA" will be +interpreted as NA. +</p> + + +<h3>Value</h3> + +<p>SparkDataFrame +</p> + + +<h3>Note</h3> + +<p>read.df since 1.4.0 +</p> +<p>loadDF since 1.6.0 +</p> + + +<h3>See Also</h3> + +<p><a href="read.json.html">read.json</a> +</p> + + +<h3>Examples</h3> + +<pre><code class="r">## Not run: +##D sparkR.session() +##D df1 <- read.df("path/to/file.json", source = "json") +##D schema <- structType(structField("name", "string"), +##D structField("info", "map<string,double>")) +##D df2 <- read.df(mapTypeJsonPath, "json", schema, multiLine = TRUE) +##D df3 <- loadDF("data/test_table", "parquet", mergeSchema = "true") +##D stringSchema <- "name STRING, info MAP<STRING, DOUBLE>" +##D df4 <- read.df(mapTypeJsonPath, "json", stringSchema, multiLine = TRUE) +## End(Not run) +</code></pre> + + +<hr /><div style="text-align: center;">[Package <em>SparkR</em> version 2.3.0 <a href="00Index.html">Index</a>]</div> +</body></html> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org