Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-147845919
From the above discussion, we can see that it's easy to get wrong, I also
+1 for close this.
---
If your project is set up for it, you can reply to this email and have
Github user HyukjinKwon closed the pull request at:
https://github.com/apache/spark/pull/8718
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-146380223
@liancheng Would this casting check be unsafe?
I came across Parquet downcasting check with the actual value
```java
public static int checkedCast(long
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143194884
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143194792
[Test build #43020 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43020/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143194885
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143175194
[Test build #43020 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43020/consoleFull)
for PR 8718 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143180019
[Test build #43022 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43022/consoleFull)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143182663
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143182633
[Test build #43022 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43022/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143182661
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143174139
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143174116
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143178132
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143178048
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-143179571
In this commit, it only deals with numbers. I removed the roundtrip in cast
and It only supports comparisons among other numeric types except `Decimal` and
between
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142654303
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142660183
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142660104
[Test build #42910 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42910/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142660186
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142654280
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142653913
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142656043
[Test build #42910 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42910/consoleFull)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-142454228
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-141960719
It looks the original case became a downcast `Decimal(10, 0)` to
`Decimal(7, 2)` which seems when scale and precision of the latter are less
than the former, rather
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-141679720
Yes I will do so.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-141275978
@HyukjinKwon How about only deal with cases that do widening? E.g. int to
double, float to double, long to decimal, etc. The original case mentioned in
SPARK-9182 is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140692487
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140692464
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140693582
It is OK for JDBC but for Parquet and ORC, it looks the conversion from
`StringType` to `NumericType` are not safe.
When the field type is `StringType`,
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140694505
[Test build #42534 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42534/consoleFull)
for PR 8718 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140720503
[Test build #42534 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42534/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140720612
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140720615
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140614859
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140614868
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140615136
[Test build #42523 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42523/consoleFull)
for PR 8718 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140630773
[Test build #42523 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42523/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140631028
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140631024
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140569493
It look ok to JDBC datasource tough. I am wondering if conversion from
`NumericType` to `StringType` should be prevented or treated for each
differently as it looks
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140295980
Does this actually work? Roundtrip in cast doesn't always end up having the
same value (e.g. casting a double to int and back loses precision)
---
If your project is set
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140369174
[Test build #42487 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42487/consoleFull)
for PR 8718 at commit
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140368436
I think in that cast it does not create any condition as `double -> int ->
double` means, the field type is int and the given value type is double. Losing
precision
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140368866
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140368891
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140401412
[Test build #42487 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42487/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140401583
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-140401582
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139517156
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139547415
[Test build #42334 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42334/console)
for PR 8718 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139547603
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139547604
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139518368
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/8718
[SPARK-9182][SQL] Cast filters are not passed through to datasources
As mentioned in https://issues.apache.org/jira/browse/SPARK-9182,
Some casts filters are not passing to datasources
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139518393
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139518413
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8718#issuecomment-139519459
[Test build #42334 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42334/consoleFull)
for PR 8718 at commit
58 matches
Mail list logo
