[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user guoxu1231 commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-162417597 BTW, There is a JIRA to suggest user to prefer using c['column'] instead of c.column. @davies, we encountered similar issue, could you paste the JIRA number

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user davies commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-157095257 @viirya This PR only help a two corner cases, and they are not blocker (could be easily workaround), I'd like to not fix these (avoid the complexity). If we merge this

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-157030709 ping @davies Any more comments? Or should we close this pr? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya closed the pull request at: https://github.com/apache/spark/pull/8934 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-157230557 @davies Yes. Thanks. I close this one now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147303344 ping @davies How about this updated version? Or you still think it is not worth due to the possible performance regressions? As in this version it only looks for `count`

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147027117 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/8934#discussion_r41692322 --- Diff: python/pyspark/sql/types.py --- @@ -1209,6 +1219,12 @@ def __new__(self, *args, **kwargs): else: raise ValueError("No

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147027121 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147028783 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147028766 [Test build #43513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43513/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147028782 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-147027398 [Test build #43513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43513/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146302967 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146302941 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146303673 [Test build #43338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43338/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146308987 [Test build #43338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43338/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146309175 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146309211 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146138449 [Test build #43325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43325/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146143887 [Test build #43325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43325/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146143957 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146143958 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146136395 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146136421 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146260317 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146260284 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146264091 [Test build #43328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43328/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146264095 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146262065 [Test build #43328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43328/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146264096 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146071744 @davies Yes. Currently as reported in the JIRA, using `x.asDict()["count"]` instead of `x.count` can be to work around this. But I think it would be better to avoid

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072217 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072238 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072983 [Test build #43317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43317/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072943 [Test build #43317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43317/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072985 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-146072984 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/8934#discussion_r41346674 --- Diff: python/pyspark/sql/types.py --- @@ -1209,6 +1219,12 @@ def __new__(self, *args, **kwargs): else: raise ValueError("No

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/8934#discussion_r41347200 --- Diff: python/pyspark/sql/types.py --- @@ -1189,6 +1189,16 @@ class Row(tuple): >>> Person("Alice", 11)

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user davies commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-145609363 @viirya It will be great we can fix it magically. I'm worried that the current approach will introduce some performance regressions. As we always have a way to

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8934#discussion_r41173178 --- Diff: python/pyspark/sql/types.py --- @@ -1189,6 +1189,16 @@ class Row(tuple): >>> Person("Alice", 11)

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8934#discussion_r41172960 --- Diff: python/pyspark/sql/types.py --- @@ -1209,6 +1219,12 @@ def __new__(self, *args, **kwargs): else: raise ValueError("No

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-145489665 ping @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144103835 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144112736 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144101807 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144101851 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144103683 [Test build #43080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43080/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144112765 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144119798 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144103826 [Test build #43080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43080/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144103832 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144113041 [Test build #43096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43096/consoleFull) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144119797 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-144119646 [Test build #43096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43096/console) for PR 8934 at commit

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/8934 [SPARK-10852][PySpark][SQL] Override built-in methods for special column names JIRA: https://issues.apache.org/jira/browse/SPARK-10852 For few special columns such as `count` and `index`,