Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20909
Yea, I know the hidden output in the console and I believe that's a known
issue. In my case, I made such change before -
https://github.com/apache/spark/pull/20487. Also see the discussion in
https://github.com/apache/spark/pull/20465.
The thing is, it needs duplicated changes to print out the warnings and
that's why I have been hesitant to fix related code paths.
Actually, I was thinking we should resemble what we do in `streaming.py` to
skip the doctests although I haven't taken a close look to check if we can
control function level yet.
I know we use `# doctest: +SKIP` here and there in particular with Pandas /
Arrow. I think basically we should remove this and do the same things to test
them when possible.
I am sure on this too (and told few committers before that I am thinking in
this way). Let me cc @cloud-fan, @ueshin and @BryanCutler FYI.
For the best, can you investigate and try to explicitly skip some doctests
conditionally? For console output from our test script, I think we can do this
separately (but please leave a comment as a todo or JIRA).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
