GitHub user drewrobb opened a pull request:
https://github.com/apache/spark/pull/15526
[SPARK-17986] [ML] SQLTransformer should remove temporary tables
## What changes were proposed in this pull request?
A call to the method `SQLTransformer.transform` previously would create a
temporary table and never delete it. This change adds a call to
`dropTempView()` that deletes this temporary table before returning the result
so that the table will not remain in spark's table catalog. Because `tableName`
is randomized and not exposed, there should be no expected use of this table
outside of the `transform` method.
## How was this patch tested?
A single new assertion was added to the existing test of the
`SQLTransformer.transform` method that all temporary tables are removed.
Without the corresponding code change, this new assertion fails. I am not aware
of any circumstances in which removing this temporary view would be bad for
performance or correctness in other ways, but some expertise here would be
helpful.
Please review
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark before
opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/drewrobb/spark SPARK-17986
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15526.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15526
----
commit d5c3b419942f1d3b9af265b540a9404d3e8295df
Author: Drew Robb <drewr...@gmail.com>
Date: 2016-10-18T03:32:55Z
SQLTransformer should remove temporary tables
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
