[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83359/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83360/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19646 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19646 @ueshin and @HyukjinKwon this allows Spark to read non-arrow Pandas timestamps to TimestampTypes instead of long values, but there are a couple things to note. I did the conversion with numpy because we can not make changes to the input pandas.DataFrame and making a copy is too expensive. When `to_records()` is called, the pdf is changed to numpy records, and that is where the check/conversion is done. For date columns, if it has a dtype of `datetime64[D]` or is a datetime object, then Spark correctly interprets to DateType. Please take a look when you can, thanks! cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19646 **[Test build #83361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83361/testReport)** for PR 19646 at commit [`839bb50`](https://github.com/apache/spark/commit/839bb5076ceeb297299e8e4c408cf437979b2bb9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19646 **[Test build #83361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83361/testReport)** for PR 19646 at commit [`839bb50`](https://github.com/apache/spark/commit/839bb5076ceeb297299e8e4c408cf437979b2bb9). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83361/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83378/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19646 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83379/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19646 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19646 **[Test build #83380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83380/testReport)** for PR 19646 at commit [`3944b5c`](https://github.com/apache/spark/commit/3944b5ca3f2ac588ee4a997ca55c10ed1456c7cf). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83380/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19646 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19646 **[Test build #83380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83380/testReport)** for PR 19646 at commit [`3944b5c`](https://github.com/apache/spark/commit/3944b5ca3f2ac588ee4a997ca55c10ed1456c7cf). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19646 @BryanCutler is this tagged to the right JIRA? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19646: [SPARK-22147][PYTHON] Fix for createDataFrame from panda...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19646 I think this should be linked to SPARK-22417. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
