[jira] [Updated] (SPARK-14883) Fix wrong R examples and make them up-to-date
[
https://issues.apache.org/jira/browse/SPARK-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivaram Venkataraman updated SPARK-14883:
--
Assignee: Dongjoon Hyun
> Fix wrong R examples and make them up-to-date
> -
>
> Key: SPARK-14883
> URL: https://issues.apache.org/jira/browse/SPARK-14883
> Project: Spark
> Issue Type: Bug
> Components: Documentation, Examples
>Reporter: Dongjoon Hyun
>Assignee: Dongjoon Hyun
> Fix For: 2.0.0
>
>
> This issue aims to fix some errors in R examples and make them up-to-date in
> docs and example modules.
> - Remove the wrong usage of map. We need to use `lapply` in `SparkR` if
> needed. However, `lapply` is private now. The correct usage will be added
> later.
> {code}
> -teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
> ...
> {code}
> - Fix the wrong example in Section `Generic Load/Save Functions` of
> `docs/sql-programming-guide.md` for consistency.
> {code}
> -df <- loadDF(sqlContext, "people.parquet")
> -saveDF(select(df, "name", "age"), "namesAndAges.parquet")
> +df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
> +write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
> {code}
> - Fix datatypes in `sparkr.md`.
> {code}
> -# |-- age: integer (nullable = true)
> +# |-- age: long (nullable = true)
> {code}
> {code}
> -## DataFrame[eruptions:double, waiting:double]
> +## SparkDataFrame[eruptions:double, waiting:double]
> {code}
> - Update data results
> {code}
> head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
> ## waiting count
> -##1 8113
> -##2 60 6
> -##3 68 1
> +##1 70 4
> +##2 67 1
> +##3 69 2
> {code}
> - Replace deprecated functions: jsonFile -> read.json, parquetFile ->
> read.parquet
> {code}
> df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
> Warning message:
> 'jsonFile' is deprecated.
> Use 'read.json' instead.
> See help("Deprecated")
> {code}
> - Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
> saveAsParquetFile -> write.parquet
> - Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
> `data-manipulation.R`.
> - Other minor syntax fixes and typos.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14883) Fix wrong R examples and make them up-to-date
[
https://issues.apache.org/jira/browse/SPARK-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-14883:
--
Description:
This issue aims to fix some errors in R examples and make them up-to-date in
docs and example modules.
- Remove the wrong usage of map. We need to use `lapply` in `SparkR` if needed.
However, `lapply` is private now. The correct usage will be added later.
{code}
-teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
...
{code}
- Fix the wrong example in Section `Generic Load/Save Functions` of
`docs/sql-programming-guide.md` for consistency.
{code}
-df <- loadDF(sqlContext, "people.parquet")
-saveDF(select(df, "name", "age"), "namesAndAges.parquet")
+df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
+write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
{code}
- Fix datatypes in `sparkr.md`.
{code}
-# |-- age: integer (nullable = true)
+# |-- age: long (nullable = true)
{code}
{code}
-## DataFrame[eruptions:double, waiting:double]
+## SparkDataFrame[eruptions:double, waiting:double]
{code}
- Update data results
{code}
head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
## waiting count
-##1 8113
-##2 60 6
-##3 68 1
+##1 70 4
+##2 67 1
+##3 69 2
{code}
- Replace deprecated functions: jsonFile -> read.json, parquetFile ->
read.parquet
{code}
df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
Warning message:
'jsonFile' is deprecated.
Use 'read.json' instead.
See help("Deprecated")
{code}
- Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
saveAsParquetFile -> write.parquet
- Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
`data-manipulation.R`.
- Other minor syntax fixes and typos.
was:
This issue aims to fix some errors in R examples and make them up-to-date in
docs and example modules.
- Fix the wrong usage of map. We need to use `lapply` if needed. However, the
usage of `lapply` also needs to be reviewed since it's private.
{code}
-teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
+teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name) })
{code}
- Fix the wrong example in Section `Generic Load/Save Functions` of
`docs/sql-programming-guide.md` for consistency.
{code}
-df <- loadDF(sqlContext, "people.parquet")
-saveDF(select(df, "name", "age"), "namesAndAges.parquet")
+df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
+write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
{code}
- Fix datatypes in `sparkr.md`.
{code}
-# |-- age: integer (nullable = true)
+# |-- age: long (nullable = true)
{code}
{code}
-## DataFrame[eruptions:double, waiting:double]
+## SparkDataFrame[eruptions:double, waiting:double]
{code}
- Update data results
{code}
head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
## waiting count
-##1 8113
-##2 60 6
-##3 68 1
+##1 70 4
+##2 67 1
+##3 69 2
{code}
- Replace deprecated functions: jsonFile -> read.json, parquetFile ->
read.parquet
{code}
df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
Warning message:
'jsonFile' is deprecated.
Use 'read.json' instead.
See help("Deprecated")
{code}
- Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
saveAsParquetFile -> write.parquet
- Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
`data-manipulation.R`.
- Other minor syntax fixes and typos.
> Fix wrong R examples and make them up-to-date
> -
>
> Key: SPARK-14883
> URL: https://issues.apache.org/jira/browse/SPARK-14883
> Project: Spark
> Issue Type: Bug
> Components: Documentation, Examples
>Reporter: Dongjoon Hyun
>
> This issue aims to fix some errors in R examples and make them up-to-date in
> docs and example modules.
> - Remove the wrong usage of map. We need to use `lapply` in `SparkR` if
> needed. However, `lapply` is private now. The correct usage will be added
> later.
> {code}
> -teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
> ...
> {code}
> - Fix the wrong example in Section `Generic Load/Save Functions` of
> `docs/sql-programming-guide.md` for consistency.
> {code}
> -df <- loadDF(sqlContext, "people.parquet")
> -saveDF(select(df, "name", "age"), "namesAndAges.parquet")
> +df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
> +write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
> {code}
> - Fix datatypes in `sparkr.md`.
> {code}
> -# |-- age: integer (nullable = true)
> +# |-- age: long (nullable = true)
> {code}
> {code}
> -## DataFrame[eruptions:double,
[jira] [Updated] (SPARK-14883) Fix wrong R examples and make them up-to-date
[
https://issues.apache.org/jira/browse/SPARK-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-14883:
--
Description:
This issue aims to fix some errors in R examples and make them up-to-date in
docs and example modules.
- Fix the wrong usage of map. We need to use `lapply` if needed. However, the
usage of `lapply` also needs to be reviewed since it's private.
{code}
-teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
+teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name) })
{code}
- Fix the wrong example in Section `Generic Load/Save Functions` of
`docs/sql-programming-guide.md` for consistency.
{code}
-df <- loadDF(sqlContext, "people.parquet")
-saveDF(select(df, "name", "age"), "namesAndAges.parquet")
+df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
+write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
{code}
- Fix datatypes in `sparkr.md`.
{code}
-# |-- age: integer (nullable = true)
+# |-- age: long (nullable = true)
{code}
{code}
-## DataFrame[eruptions:double, waiting:double]
+## SparkDataFrame[eruptions:double, waiting:double]
{code}
- Update data results
{code}
head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
## waiting count
-##1 8113
-##2 60 6
-##3 68 1
+##1 70 4
+##2 67 1
+##3 69 2
{code}
- Replace deprecated functions: jsonFile -> read.json, parquetFile ->
read.parquet
{code}
df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
Warning message:
'jsonFile' is deprecated.
Use 'read.json' instead.
See help("Deprecated")
{code}
- Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
saveAsParquetFile -> write.parquet
- Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
`data-manipulation.R`.
- Other minor syntax fixes and typos.
was:
This issue aims to fix some errors in R examples and make them up-to-date in
docs and example modules.
- Fix the wrong usage of map. We need to use `lapply` if needed. However, the
usage of `lapply` also needs to be reviewed since it's private.
{code}
-teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
+teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name) })
{code}
- Fix the wrong example in Section `Generic Load/Save Functions` of
`docs/sql-programming-guide.md` for consistency.
{code}
-df <- loadDF(sqlContext, "people.parquet")
-saveDF(select(df, "name", "age"), "namesAndAges.parquet")
+df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
+write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
{code}
- Fix datatypes in `sparkr.md`.
{code}
-# |-- age: integer (nullable = true)
+# |-- age: long (nullable = true)
{code}
{code}
-## DataFrame[eruptions:double, waiting:double]
+## SparkDataFrame[eruptions:double, waiting:double]
{code}
- Update data results
{code}
head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
## waiting count
-##1 8113
-##2 60 6
-##3 68 1
+##1 70 4
+##2 67 1
+##3 69 2
{code}
- Replace deprecated functions: jsonFile -> read.json, parquetFile ->
read.parquet
{code}
df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
Warning message:
'jsonFile' is deprecated.
Use 'read.json' instead.
See help("Deprecated")
{code}
- Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
saveAsParquetFile -> write.parquet
- Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
`data-manipulation.R`.
- Other minor syntax fixes and typos.
> Fix wrong R examples and make them up-to-date
> -
>
> Key: SPARK-14883
> URL: https://issues.apache.org/jira/browse/SPARK-14883
> Project: Spark
> Issue Type: Bug
> Components: Documentation, Examples
>Reporter: Dongjoon Hyun
>
> This issue aims to fix some errors in R examples and make them up-to-date in
> docs and example modules.
> - Fix the wrong usage of map. We need to use `lapply` if needed. However, the
> usage of `lapply` also needs to be reviewed since it's private.
> {code}
> -teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
> +teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name)
> })
> {code}
> - Fix the wrong example in Section `Generic Load/Save Functions` of
> `docs/sql-programming-guide.md` for consistency.
> {code}
> -df <- loadDF(sqlContext, "people.parquet")
> -saveDF(select(df, "name", "age"), "namesAndAges.parquet")
> +df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
> +write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
> {code}
> - Fix datatypes in `sparkr.md`.
>
[jira] [Updated] (SPARK-14883) Fix wrong R examples and make them up-to-date
[
https://issues.apache.org/jira/browse/SPARK-14883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-14883:
--
Description:
This issue aims to fix some errors in R examples and make them up-to-date in
docs and example modules.
- Fix the wrong usage of map. We need to use `lapply` if needed. However, the
usage of `lapply` also needs to be reviewed since it's private.
{code}
-teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
+teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name) })
{code}
- Fix the wrong example in Section `Generic Load/Save Functions` of
`docs/sql-programming-guide.md` for consistency.
{code}
-df <- loadDF(sqlContext, "people.parquet")
-saveDF(select(df, "name", "age"), "namesAndAges.parquet")
+df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
+write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
{code}
- Fix datatypes in `sparkr.md`.
{code}
-# |-- age: integer (nullable = true)
+# |-- age: long (nullable = true)
{code}
{code}
-## DataFrame[eruptions:double, waiting:double]
+## SparkDataFrame[eruptions:double, waiting:double]
{code}
- Update data results
{code}
head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
## waiting count
-##1 8113
-##2 60 6
-##3 68 1
+##1 70 4
+##2 67 1
+##3 69 2
{code}
- Replace deprecated functions: jsonFile -> read.json, parquetFile ->
read.parquet
{code}
df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
Warning message:
'jsonFile' is deprecated.
Use 'read.json' instead.
See help("Deprecated")
{code}
- Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
saveAsParquetFile -> write.parquet
- Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
`data-manipulation.R`.
- Other minor syntax fixes and typos.
was:
This issue aims to fix some errors in R examples and make them up-to-date in
docs and example modules.
- Fix the wrong usage of map. We need to use `lapply` if needed. However, the
usage of `lapply` also needs to be reviewed since it's private.
{code}
-teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
+teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name) })
{code}
- Fix the wrong example in Section `Generic Load/Save Functions` of
`docs/sql-programming-guide.md` for consistency.
{code}
-df <- loadDF(sqlContext, "people.parquet")
-saveDF(select(df, "name", "age"), "namesAndAges.parquet")
+df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
+write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
{code}
- Replace deprecated functions: jsonFile -> read.json, parquetFile ->
read.parquet
{code}
df <- jsonFile(sqlContext, "examples/src/main/resources/people.json")
Warning message:
'jsonFile' is deprecated.
Use 'read.json' instead.
See help("Deprecated")
{code}
- Use up-to-date R-like functions: loadDF -> read.df, saveDF -> write.df,
saveAsParquetFile -> write.parquet
- Replace `SparkR DataFrame` with `SparkDataFrame` in `dataframe.R` and
`data-manipulation.R`.
- Other minor syntax fixes and typos.
> Fix wrong R examples and make them up-to-date
> -
>
> Key: SPARK-14883
> URL: https://issues.apache.org/jira/browse/SPARK-14883
> Project: Spark
> Issue Type: Bug
> Components: Documentation, Examples
>Reporter: Dongjoon Hyun
>
> This issue aims to fix some errors in R examples and make them up-to-date in
> docs and example modules.
> - Fix the wrong usage of map. We need to use `lapply` if needed. However, the
> usage of `lapply` also needs to be reviewed since it's private.
> {code}
> -teenNames <- map(teenagers, function(p) { paste("Name:", p$name)})
> +teenNames <- SparkR:::lapply(teenagers, function(p) { paste("Name:", p$name)
> })
> {code}
> - Fix the wrong example in Section `Generic Load/Save Functions` of
> `docs/sql-programming-guide.md` for consistency.
> {code}
> -df <- loadDF(sqlContext, "people.parquet")
> -saveDF(select(df, "name", "age"), "namesAndAges.parquet")
> +df <- read.df(sqlContext, "examples/src/main/resources/users.parquet")
> +write.df(select(df, "name", "favorite_color"), "namesAndFavColors.parquet")
> {code}
> - Fix datatypes in `sparkr.md`.
> {code}
> -# |-- age: integer (nullable = true)
> +# |-- age: long (nullable = true)
> {code}
> {code}
> -## DataFrame[eruptions:double, waiting:double]
> +## SparkDataFrame[eruptions:double, waiting:double]
> {code}
> - Update data results
> {code}
> head(summarize(groupBy(df, df$waiting), count = n(df$waiting)))
> ## waiting count
> -##1 8113
> -##2 60 6
> -##3 68 1
> +##1 70 4
> +##2 67 1
> +##3 69 2
> {code}
> -
