[
https://issues.apache.org/jira/browse/SPARK-36051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-36051:
---------------------------------
Description:
At
https://github.com/apache/spark/tree/master/docs#automatically-rebuilding-api-docs,
{code}
cd "$SPARK_HOME/python/docs"
find .. -type f -name '*.py' \
| entr -s 'make html && cp -r _build/html/. ../../docs/api/python'
{code}
is documented. However, this doesn't work very well:
1. It doesn't detect the changes in .rst files. But PySpark internally
generates rst files so we can't just simply include it in the detection.
Otherwise, it goes to an infinite loop
2. During PySpark documentation generation, it launches some jobs to generate
plot images now. This is broken with {{entr}} command, and the job fails. Seems
like it's related to how {{entr}} creates the process internally.
was:
At
https://github.com/apache/spark/tree/master/docs#automatically-rebuilding-api-docs,
{code}
cd "$SPARK_HOME/python/docs"
find .. -type f -name '*.py' \
| entr -s 'make html && cp -r _build/html/. ../../docs/api/python'
{code}
we should also check .rst files
> Remove automatic update of documentation build in the guides
> ------------------------------------------------------------
>
> Key: SPARK-36051
> URL: https://issues.apache.org/jira/browse/SPARK-36051
> Project: Spark
> Issue Type: Improvement
> Components: docs, PySpark
> Affects Versions: 3.2.0
> Reporter: Hyukjin Kwon
> Priority: Trivial
>
> At
> https://github.com/apache/spark/tree/master/docs#automatically-rebuilding-api-docs,
>
> {code}
> cd "$SPARK_HOME/python/docs"
> find .. -type f -name '*.py' \
> | entr -s 'make html && cp -r _build/html/. ../../docs/api/python'
> {code}
> is documented. However, this doesn't work very well:
> 1. It doesn't detect the changes in .rst files. But PySpark internally
> generates rst files so we can't just simply include it in the detection.
> Otherwise, it goes to an infinite loop
> 2. During PySpark documentation generation, it launches some jobs to generate
> plot images now. This is broken with {{entr}} command, and the job fails.
> Seems like it's related to how {{entr}} creates the process internally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
