[
https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xusen Yin updated SPARK-11337:
------------------------------
Description:
The example code in the user guide is embedded in the markdown and hence it is
not easy to test. It would be nice to automatically test them. This JIRA is to
discuss options to automate example code testing and see what we can do in
Spark 1.6.
One option I propose is to move actual example code to spark/examples and test
compilation in Jenkins builds. Then in the markdown, we can reference part of
the code to show in the user guide. This requires adding a Jekyll tag that is
similar to
https://github.com/jekyll/jekyll/blob/master/lib/jekyll/tags/include.rb, e.g.,
called include_example.
{code}
{% include_example scala ml.KMeansExample guide %}
{code}
Jekyll will find
`examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala` and
pick code blocks marked "example" and put them under `{% highlight %}` in the
markdown. We can discuss the syntax for marker comments.
Sub-tasks are created to move example code from user guide to `examples/`.
*self-check list for contributors in this JIRA*
# Be sure to match Scala/Java/Python code style guide. If unsure of a code
style, please refer to other merged example code under examples/.
# Remove useless imports
# It's better to have a side-effect operation at the end of each example code,
usually it's a {code}print(...){code}.
# Make sure the code example is runnable without error.
# After finishing code migration, use {code}cd docs; SKIP_API=1 jekyll
serve{code} the check the webpage at http://127.0.0.1:4000 to see whether the
generated html looks good.
was:
The example code in the user guide is embedded in the markdown and hence it is
not easy to test. It would be nice to automatically test them. This JIRA is to
discuss options to automate example code testing and see what we can do in
Spark 1.6.
One option I propose is to move actual example code to spark/examples and test
compilation in Jenkins builds. Then in the markdown, we can reference part of
the code to show in the user guide. This requires adding a Jekyll tag that is
similar to
https://github.com/jekyll/jekyll/blob/master/lib/jekyll/tags/include.rb, e.g.,
called include_example.
{code}
{% include_example scala ml.KMeansExample guide %}
{code}
Jekyll will find
`examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala` and
pick code blocks marked "example" and put them under `{% highlight %}` in the
markdown. We can discuss the syntax for marker comments.
Sub-tasks are created to move example code from user guide to `examples/`.
> Make example code in user guide testable
> ----------------------------------------
>
> Key: SPARK-11337
> URL: https://issues.apache.org/jira/browse/SPARK-11337
> Project: Spark
> Issue Type: Umbrella
> Components: Documentation
> Reporter: Xiangrui Meng
> Assignee: Xusen Yin
> Priority: Critical
>
> The example code in the user guide is embedded in the markdown and hence it
> is not easy to test. It would be nice to automatically test them. This JIRA
> is to discuss options to automate example code testing and see what we can do
> in Spark 1.6.
> One option I propose is to move actual example code to spark/examples and
> test compilation in Jenkins builds. Then in the markdown, we can reference
> part of the code to show in the user guide. This requires adding a Jekyll tag
> that is similar to
> https://github.com/jekyll/jekyll/blob/master/lib/jekyll/tags/include.rb,
> e.g., called include_example.
> {code}
> {% include_example scala ml.KMeansExample guide %}
> {code}
> Jekyll will find
> `examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala`
> and pick code blocks marked "example" and put them under `{% highlight %}` in
> the markdown. We can discuss the syntax for marker comments.
> Sub-tasks are created to move example code from user guide to `examples/`.
> *self-check list for contributors in this JIRA*
> # Be sure to match Scala/Java/Python code style guide. If unsure of a code
> style, please refer to other merged example code under examples/.
> # Remove useless imports
> # It's better to have a side-effect operation at the end of each example
> code, usually it's a {code}print(...){code}.
> # Make sure the code example is runnable without error.
> # After finishing code migration, use {code}cd docs; SKIP_API=1 jekyll
> serve{code} the check the webpage at http://127.0.0.1:4000 to see whether the
> generated html looks good.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
