[REPORT] ORC - April 2022

2022-04-04 Thread Dongjoon Hyun
## Description: The mission of ORC is the creation and maintenance of software related to the smallest, fastest columnar storage for Hadoop workloads. ## Issues: There are no issues requiring board attention. ## Membership Data: Apache ORC was founded 2015-04-21 (7 years ago) There are currently

Re: [DRAFT] ORC Board Report for April 2022

2022-04-04 Thread Dongjoon Hyun
Hi, All. Thanks to William and Yiqun. We verified and fixed them. It seems that we need to release Apache ORC 1.6.14 together. Let me prepare it after 1.7.4 release. For the ORC Board Report, I'll finalize and send it with those updates. Thanks, Dongjoon On Sun, Apr 3, 2022 at 1:26 PM

[GitHub] [orc] dongjoon-hyun commented on pull request #1080: ORC-1147: Use `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1080: URL: https://github.com/apache/orc/pull/1080#issuecomment-1088283283 Thank sounds better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [orc] dongjoon-hyun commented on issue #1075: ORC 1.7.4-SNAPSHOT fails with Iceberg Nan count tests

2022-04-04 Thread GitBox
dongjoon-hyun commented on issue #1075: URL: https://github.com/apache/orc/issues/1075#issuecomment-1088282881 If you get the results, please share them to the dev mailing list, @williamhyun . -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [orc] dongjoon-hyun commented on issue #1075: ORC 1.7.4-SNAPSHOT fails with Iceberg Nan count tests

2022-04-04 Thread GitBox
dongjoon-hyun commented on issue #1075: URL: https://github.com/apache/orc/issues/1075#issuecomment-1088274728 I also quickly re-checked the Iceberg test. So, `TestMetricsRowGroupFilter` passed except the following one mentioned by @guiyanakuang . ``` shouldRead =

[GitHub] [orc] guiyanakuang commented on pull request #1080: ORC-1147: Use `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write

2022-04-04 Thread GitBox
guiyanakuang commented on PR #1080: URL: https://github.com/apache/orc/pull/1080#issuecomment-1088267454 > Oh, @guiyanakuang . This is not correct, @guiyanakuang . > > There exists multiple NaN values. Double.NaN is just one of them. Maybe I should use the JDK's own method `new

[GitHub] [orc] dongjoon-hyun commented on pull request #1080: ORC-1147: Use `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1080: URL: https://github.com/apache/orc/pull/1080#issuecomment-1088257901 Please note that IEEE Standard defines NaN as range, not a single value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [orc] dongjoon-hyun merged pull request #1078: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun merged PR #1078: URL: https://github.com/apache/orc/pull/1078 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [orc] guiyanakuang opened a new pull request, #1080: ORC-1147: Use `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write

2022-04-04 Thread GitBox
guiyanakuang opened a new pull request, #1080: URL: https://github.com/apache/orc/pull/1080 ### What changes were proposed in this pull request? This pr is aimed at using `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write.

[jira] [Created] (ORC-1147) Use `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write

2022-04-04 Thread Yiqun Zhang (Jira)
Yiqun Zhang created ORC-1147: Summary: Use `Objects.equals(dstas.getSum(), Double.NaN)` instead of `isFinite` to determine if there is a NaN write Key: ORC-1147 URL: https://issues.apache.org/jira/browse/ORC-1147

[GitHub] [orc] dongjoon-hyun merged pull request #1079: ORC-1016: Use `openssl@1.1` in GitHub Action MacOS CIs

2022-04-04 Thread GitBox
dongjoon-hyun merged PR #1079: URL: https://github.com/apache/orc/pull/1079 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [orc] dongjoon-hyun opened a new pull request, #1079: ORC-1016: Use `openssl@1.1` in GitHub Action MacOS CIs

2022-04-04 Thread GitBox
dongjoon-hyun opened a new pull request, #1079: URL: https://github.com/apache/orc/pull/1079 ### What changes were proposed in this pull request? This is a backport of ORC-1016 because `branch-1.6` seems to hit this issue in the following PR. -

[GitHub] [orc] dongjoon-hyun commented on pull request #1078: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1078: URL: https://github.com/apache/orc/pull/1078#issuecomment-1088210328 cc @guiyanakuang . This will be merged with your authorship after passing CIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [orc] dongjoon-hyun opened a new pull request, #1078: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun opened a new pull request, #1078: URL: https://github.com/apache/orc/pull/1078 ### What changes were proposed in this pull request? This pr is aimed at checking whether the float category statistic sum has a finite value. ### Why are the changes needed?

[GitHub] [orc] williamhyun commented on issue #1075: ORC 1.7.4-SNAPSHOT fails with Iceberg Nan count tests

2022-04-04 Thread GitBox
williamhyun commented on issue #1075: URL: https://github.com/apache/orc/issues/1075#issuecomment-1088209651 Thank you @guiyanakuang ! I am retesting on both Iceberg and Spark. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [orc] dongjoon-hyun commented on pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1077: URL: https://github.com/apache/orc/pull/1077#issuecomment-1088207106 Thank you, @guiyanakuang . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [orc] guiyanakuang commented on pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
guiyanakuang commented on PR #1077: URL: https://github.com/apache/orc/pull/1077#issuecomment-1088196218 Thanks @dongjoon-hyun . I checked the branch-1.6 and confirmed it has the same problem. This pr can just cherry pick and fix it without any conflicts. -- This is an automated message

[GitHub] [orc] dongjoon-hyun commented on pull request #1073: ORC-969: [C++] Evaluate SearchArguments using file and stripe level stats

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1073: URL: https://github.com/apache/orc/pull/1073#issuecomment-1088162263 Oh, that looks like a good improvement item in Java. Could you please file a JIRA for that if we don't have it yet, @coderex2522 ? -- This is an automated message from the Apache

[GitHub] [orc] dongjoon-hyun commented on pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1077: URL: https://github.com/apache/orc/pull/1077#issuecomment-1087846051 BTW, @guiyanakuang . Could you check branch-1.6 too when you have a time? We backported the following in branch-1.6. I guess we might have the same issue there. -

[GitHub] [orc] dongjoon-hyun commented on pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1077: URL: https://github.com/apache/orc/pull/1077#issuecomment-1087834344 I deleted my previous comment about the branch-1.7 test result. At a clean build, I verified this patch in branch-1.7 and backported. Thank you again, @guiyanakuang .

[GitHub] [orc] dongjoon-hyun commented on pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun commented on PR #1077: URL: https://github.com/apache/orc/pull/1077#issuecomment-1087819831 Hi, @guiyanakuang . - Could you make a backporting patch to branch-1.7, please? - This works in `main` branch but not in `branch-1.7`. We need to pass CI. -- This is an

[GitHub] [orc] dongjoon-hyun closed pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
dongjoon-hyun closed pull request #1077: ORC-1146: Float category missing check if the statistic sum is a finite value URL: https://github.com/apache/orc/pull/1077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [orc] dependabot[bot] commented on pull request #1076: Bump netty-all from 4.1.68.Final to 4.1.75.Final in /java

2022-04-04 Thread GitBox
dependabot[bot] commented on PR #1076: URL: https://github.com/apache/orc/pull/1076#issuecomment-1087767116 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me

[GitHub] [orc] dongjoon-hyun closed pull request #1076: Bump netty-all from 4.1.68.Final to 4.1.75.Final in /java

2022-04-04 Thread GitBox
dongjoon-hyun closed pull request #1076: Bump netty-all from 4.1.68.Final to 4.1.75.Final in /java URL: https://github.com/apache/orc/pull/1076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [orc] dongjoon-hyun commented on issue #1075: ORC 1.7.4-SNAPSHOT fails with Iceberg Nan count tests

2022-04-04 Thread GitBox
dongjoon-hyun commented on issue #1075: URL: https://github.com/apache/orc/issues/1075#issuecomment-1087744468 Oh, thank you, @guiyanakuang ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [orc] guiyanakuang opened a new pull request, #1077: ORC-1146: Float category missing check if the statistic sum is a finite value

2022-04-04 Thread GitBox
guiyanakuang opened a new pull request, #1077: URL: https://github.com/apache/orc/pull/1077 ### What changes were proposed in this pull request? This pr is aimed at checking whether the float category statistic sum has a finite value. ### Why are the changes needed?

[jira] [Created] (ORC-1146) Float category does not check if the statistic sum is a finite value

2022-04-04 Thread Yiqun Zhang (Jira)
Yiqun Zhang created ORC-1146: Summary: Float category does not check if the statistic sum is a finite value Key: ORC-1146 URL: https://issues.apache.org/jira/browse/ORC-1146 Project: ORC Issue

[GitHub] [orc] guiyanakuang commented on issue #1075: ORC 1.7.4-SNAPSHOT fails with Iceberg Nan count tests

2022-04-04 Thread GitBox
guiyanakuang commented on issue #1075: URL: https://github.com/apache/orc/issues/1075#issuecomment-1087526984 Thanks to capturing these failed tests, @williamhyun I conducted some investigation. Prior to #1055, it was just coincidence that these tests would pass, after #1055 was fixed

[GitHub] [orc] dependabot[bot] opened a new pull request, #1076: Bump netty-all from 4.1.68.Final to 4.1.75.Final in /java

2022-04-04 Thread GitBox
dependabot[bot] opened a new pull request, #1076: URL: https://github.com/apache/orc/pull/1076 Bumps [netty-all](https://github.com/netty/netty) from 4.1.68.Final to 4.1.75.Final. Commits https://github.com/netty/netty/commit/5f1802ac65d63076616c5b3ee7a3d4151c7f3414;>5f1802a

[GitHub] [orc] coderex2522 commented on pull request #1073: ORC-969: [C++] Evaluate SearchArguments using file and stripe level stats

2022-04-04 Thread GitBox
coderex2522 commented on PR #1073: URL: https://github.com/apache/orc/pull/1073#issuecomment-1087315808 > BTW, do we have a similar code in Java side, @coderex2522 ? I've looked at the SargApplier java code in the RecordReaderImpl.java file and it doesn't implement filtering based on