[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-22 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140419964 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1760,13 +1760,39 @@ def toPandas(self): "if

[GitHub] spark issue #18945: [SPARK-21766][SQL] Convert nullable int columns to float...

2017-09-21 Thread logannc
Github user logannc commented on the issue: https://github.com/apache/spark/pull/18945 I've continued to use @HyukjinKwon 's suggestion because it should be more performant and is capable of handling it without loss of precision. I believe I've addressed your concerns by only

[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-21 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140414783 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-21 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140413579 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-21 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140412857 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark pull request #18945: [SPARK-21766][SQL] Convert nullable int columns t...

2017-09-21 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140412745 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-09-20 Thread logannc
Github user logannc commented on the issue: https://github.com/apache/spark/pull/18945 Hm. Where would I add tests? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-09-20 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140153330 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-09-20 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140148777 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-09-20 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140148164 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-09-20 Thread logannc
Github user logannc commented on a diff in the pull request: https://github.com/apache/spark/pull/18945#discussion_r140147933 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1761,12 +1761,37 @@ def toPandas(self): raise ImportError("%s\n%s" % (e.me

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-09-20 Thread logannc
Github user logannc commented on the issue: https://github.com/apache/spark/pull/18945 Sorry I feel off the face of the earth. I finally had some time to sit down and do this. I took your suggestions but implemented it a little differently. Unless I've made a dumb mistake, I think I

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-08-27 Thread logannc
Github user logannc commented on the issue: https://github.com/apache/spark/pull/18945 Sorry for the delay. Things got busy and now there is the storm in Houston. Will update this per these suggestions soon. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-08-17 Thread logannc
Github user logannc commented on the issue: https://github.com/apache/spark/pull/18945 I read the contributing guide. It said that simple changes didn't need a JIRA. Certainly this code change is quite simple, I just wasn't sure if there would be enough discussion to warrant a Jira

[GitHub] spark pull request #18945: Add option to convert nullable int columns to flo...

2017-08-14 Thread logannc
GitHub user logannc opened a pull request: https://github.com/apache/spark/pull/18945 Add option to convert nullable int columns to float columns in toPand… …as to prevent needless Exceptions during routine use. Add the `strict=True` kwarg to DataFrame.toPandas to allow