Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-212247762
Thank you for your kind reviews during last month, @rxin and @srowen .
I'm happily closing this PR. It's my bad to take so much time to close this
PR.
---
If
Github user dongjoon-hyun closed the pull request at:
https://github.com/apache/spark/pull/11868
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-212150790
It's just for a record.
In case there is no comment from @JoshRosen , I'll close this PR at tonight.
Silence means many things and seems enough for this PR.
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-212147570
Sure. As I mentioned today, I'm going to close this PR since Spark don't
want this.
I'm wondering what the intention of those testcases was.
Don't worry, @
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-212143922
Yea unfortunately the behavior is already pretty weird, and I'm not sure
adding this would actually make it less weird, so I'm in favor of just not
doing anything here.
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-212020852
Hi, @JoshRosen .
Could you take a look at this PR about Row(Double.NaN) == Row(Float.NaN)
for a second when you have some time today? I'm going to close this P
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/11868#discussion_r59162191
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -402,11 +402,13 @@ trait Row extends Serializable {
retur
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-208118258
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-208118250
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-208117787
**[Test build #55494 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55494/consoleFull)**
for PR 11868 at commit
[`1609112`](https://g
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11868#discussion_r59145545
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala ---
@@ -402,11 +402,13 @@ trait Row extends Serializable {
return false
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-208101127
**[Test build #55494 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55494/consoleFull)**
for PR 11868 at commit
[`1609112`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-201112254
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-201112087
**[Test build #54124 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54124/consoleFull)**
for PR 11868 at commit
[`bafb5ad`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-201086373
**[Test build #54124 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54124/consoleFull)**
for PR 11868 at commit
[`bafb5ad`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-201024818
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-201024823
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-201024200
**[Test build #54055 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54055/consoleFull)**
for PR 11868 at commit
[`a0e4ebe`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-200929579
**[Test build #54055 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54055/consoleFull)**
for PR 11868 at commit
[`a0e4ebe`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-28340
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-28345
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-27579
**[Test build #53799 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53799/consoleFull)**
for PR 11868 at commit
[`5e0a19c`](https://g
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-10025
I think this PR makes Spark users feel less confused by completing the
missing part of `NaN` in Spark SQL(Row).
---
If your project is set up for it, you can rep
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199982128
Oh, I see what is the point here now. @rxin , may I explain a little bit
more?
Mathematically, `NaN` equality is defined `false`. The followings are all
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199961154
[IBM
DB2](https://www.ibm.com/support/knowledgecenter/SSEPEK_10.0.0/com.ibm.db2z10.doc.sqlref/src/tpc/db2z_numericcomparisions.dita)
also says "From an SQL perspe
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199960521
Yea but if they do
row1.getFloat(1) == row2.getDouble(2), it'd ...
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199960241
For example,
[Oracle](https://docs.oracle.com/cd/B12037_01/server.101/b10759/sql_elements001.htm)
orders NaN greatest with respect to all other values, and evalua
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199959344
It's because Scala uses the standard way of Java and IEEE floating point. I
also know that NaN is always false with even other NaN.
However, it's about `Row`.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199957564
I spent a bit of time on this -- I'm actually not sure we want to change
this anymore, because Scala itsefl doesn't do this and users can always screw
up if they do field
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199956737
Hi, @rxin . Thank you again!
I made a big mistake in this PR and now I fixed it due to your advice. Now,
the followings are true.
```
scala> row_a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199954892
**[Test build #53799 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53799/consoleFull)**
for PR 11868 at commit
[`5e0a19c`](https://gi
Github user dongjoon-hyun commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199874683
Oh, thank you for pointing out that. I missed that part. Let me check that
again. I guess we can change `Row[NaN].hashCode` together in this PR.
---
If your proj
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199678899
What do we do for hash code?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199570349
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199570345
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199569904
**[Test build #53726 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53726/consoleFull)**
for PR 11868 at commit
[`6ff7ade`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11868#issuecomment-199536386
**[Test build #53726 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53726/consoleFull)**
for PR 11868 at commit
[`6ff7ade`](https://gi
37 matches
Mail list logo
