[GitHub] [orc] kumonlin commented on pull request #1609: ORC-1500: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
kumonlin commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1705859057 > Thank you for making a PR, @kumonlin . > > I have the same comment with @wgtmac . In addition, do you think you can make a unit test case for this `encoded_output` change? >

[GitHub] [orc] kumonlin commented on pull request #1609: ORC-1500: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
kumonlin commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1705856477 > Thanks for submitting a PR! > > BTW, could you please keep the PR template? done -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [orc] kumonlin commented on pull request #1609: ORC-1500: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
kumonlin commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1705858423 > The URI is the open-source libhdfspp interface of the Hadoop community, and the corresponding unit test is available. Therefore, you do not need to add the URI unit test to the

[GitHub] [orc] wgtmac commented on pull request #1609: ORC-1500: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
wgtmac commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1705811055 Actually I am a little bit confused with the title. Do you mean the hive partition which is a directory in the HDFS file path? In terms of the ORC reader, it does not know any partition

[GitHub] [orc] kumonlin commented on pull request #1609: ORC-1500: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
kumonlin commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1705840423 If the partition field is set in the Hive data set, the data content stored in the partition field is an independent path in the HDFS. Therefore, if the partition field contains special

[GitHub] [orc] zxf216 opened a new issue, #1610: Does the C++ version support write-encryption and read-decryption?

2023-09-11 Thread via GitHub
zxf216 opened a new issue, #1610: URL: https://github.com/apache/orc/issues/1610 Does the C++ version support write-encryption and read-decryption? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [orc] dependabot[bot] opened a new pull request, #1611: Bump org.apache.maven.plugins:maven-enforcer-plugin from 3.4.0 to 3.4.1 in /java

2023-09-11 Thread via GitHub
dependabot[bot] opened a new pull request, #1611: URL: https://github.com/apache/orc/pull/1611 Bumps [org.apache.maven.plugins:maven-enforcer-plugin](https://github.com/apache/maven-enforcer) from 3.4.0 to 3.4.1. Release notes Sourced from

[GitHub] [orc] dongjoon-hyun opened a new pull request, #1612: ORC-1502: Upgrade Maven to 3.9.4

2023-09-11 Thread via GitHub
dongjoon-hyun opened a new pull request, #1612: URL: https://github.com/apache/orc/pull/1612 ### What changes were proposed in this pull request? ### Why are the changes needed? ### How was this patch tested? -- This is an automated message

[GitHub] [orc] dongjoon-hyun commented on issue #1610: Does the C++ version support write-encryption and read-decryption?

2023-09-11 Thread via GitHub
dongjoon-hyun commented on issue #1610: URL: https://github.com/apache/orc/issues/1610#issuecomment-1714392468 Not yet. We have been tracking it here, #1183 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [orc] dongjoon-hyun closed issue #1610: Does the C++ version support write-encryption and read-decryption?

2023-09-11 Thread via GitHub
dongjoon-hyun closed issue #1610: Does the C++ version support write-encryption and read-decryption? URL: https://github.com/apache/orc/issues/1610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [orc] zxf216 commented on issue #1610: Does the C++ version support write-encryption and read-decryption?

2023-09-11 Thread via GitHub
zxf216 commented on issue #1610: URL: https://github.com/apache/orc/issues/1610#issuecomment-1714836132 Is there a plan to develop C++ encryption and decryption functionality now, as it was planned for version 1.9.0? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [orc] mystic-lama opened a new pull request, #1616: ORC-1504: Add lower bound check for get() in DynamicIntArray

2023-09-17 Thread via GitHub
mystic-lama opened a new pull request, #1616: URL: https://github.com/apache/orc/pull/1616 ### What changes were proposed in this pull request? The PR proposes adding check for lower bound for index passed to get() in DynamicIntArray ### Why are the changes needed?

[GitHub] [orc] dongjoon-hyun commented on issue #1583: Release Apache ORC 1.8.5

2023-08-30 Thread via GitHub
dongjoon-hyun commented on issue #1583: URL: https://github.com/apache/orc/issues/1583#issuecomment-1699539279 BTW, for me, it looks like a dev environment issue. ``` /root/orc/c++/src/ByteRLE.cc:469:36: error: implicit conversion changes signedness: 'signed char' to 'char'

[GitHub] [orc] dongjoon-hyun commented on issue #1583: Release Apache ORC 1.8.5

2023-08-30 Thread via GitHub
dongjoon-hyun commented on issue #1583: URL: https://github.com/apache/orc/issues/1583#issuecomment-1699527575 Hi, @wgtmac . You need to`reinit` the OS image first inside `branch-1.8`. ``` $ ./reinit.sh ``` -- This is an automated message from the Apache Git Service. To respond

[GitHub] [orc] wgtmac commented on issue #1583: Release Apache ORC 1.8.5

2023-08-30 Thread via GitHub
wgtmac commented on issue #1583: URL: https://github.com/apache/orc/issues/1583#issuecomment-1699487946 It seems that I have a lot of issues running the docker test on the branch-1.8. ``` tail -n10 logs/*test* ==> logs/centos7-test.log <== [INFO] No dependency problems found

[GitHub] [orc] guiyanakuang commented on issue #1583: Release Apache ORC 1.8.5

2023-08-30 Thread via GitHub
guiyanakuang commented on issue #1583: URL: https://github.com/apache/orc/issues/1583#issuecomment-1699536327 I was running the command on the branch-1.8. Judging from the error, it may be that the memory allocated by docker is insufficient. -- This is an automated message from the

[GitHub] [orc] abdulwadood97 opened a new issue, #1613: CMAKE linker error for snappy

2023-09-12 Thread via GitHub
abdulwadood97 opened a new issue, #1613: URL: https://github.com/apache/orc/issues/1613 Whenever I try to compile using "make package" within build directory. it gives this error in "nappy_ep-build-err.log" " snappy_unittest.cc:(.text+0x680b): undefined reference to

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-09-12 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1323864572 ## java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java: ## @@ -131,11 +131,21 @@ public boolean isTrackingDiskRanges() { return zcr != null; } +

[GitHub] [orc] dongjoon-hyun commented on issue #1613: CMAKE linker error for snappy

2023-09-12 Thread via GitHub
dongjoon-hyun commented on issue #1613: URL: https://github.com/apache/orc/issues/1613#issuecomment-1716983035 Hi, @abdulwadood97 . Could you describe your environment your procedure? Apache ORC maintains GitHub Action test coverage and we don't see any compilation failures in

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-09-12 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1323864572 ## java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java: ## @@ -131,11 +131,21 @@ public boolean isTrackingDiskRanges() { return zcr != null; } +

[GitHub] [orc] abdulwadood97 commented on issue #1613: CMAKE linker error for snappy

2023-09-13 Thread via GitHub
abdulwadood97 commented on issue #1613: URL: https://github.com/apache/orc/issues/1613#issuecomment-1717033223 I am using ubuntu 18.04.2 and linux kernel 5.4.0-49-generic for c++ only with AVX. The steps are as mentioned in the readme. 1. Clone. 2. Export the avx variable 3.

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-09-12 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1323864572 ## java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java: ## @@ -131,11 +131,21 @@ public boolean isTrackingDiskRanges() { return zcr != null; } +

[GitHub] [orc] zxf216 commented on issue #1183: Add C++ API for Columnar Encryption

2023-09-13 Thread via GitHub
zxf216 commented on issue #1183: URL: https://github.com/apache/orc/issues/1183#issuecomment-1717504392 > > Hi @dongjoon-hyun Do we have jira to track the work? > > @deshanxiao Not yet. @coderex2522 is working on it already. Will create JIRAs later. @coderex2522 Has the ORC

[GitHub] [orc] mystic-lama opened a new pull request, #1614: ORC-1503: Updated Maven version to 3.9.4

2023-09-13 Thread via GitHub
mystic-lama opened a new pull request, #1614: URL: https://github.com/apache/orc/pull/1614 ORC-1503: Update README.md to use Maven 3.9.4 ### What changes were proposed in this pull request? This PR aims to update documentation to use Maven 3.9.4, upgraded as part of ORC-1502

[GitHub] [orc] dongjoon-hyun commented on pull request #1609: ORC-1500: [C++] The partition field does not support English special characters

2023-09-08 Thread via GitHub
dongjoon-hyun commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1711925628 No, we need a test coverage for your new code in this PR, not some existing code in the other repository, @kumonlin . > The URI is the open-source libhdfspp interface of the Hadoop

[GitHub] [orc] guiyanakuang commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-09-07 Thread via GitHub
guiyanakuang commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1319297862 ## java/core/src/java/org/apache/orc/impl/RecordReaderUtils.java: ## @@ -131,11 +131,21 @@ public boolean isTrackingDiskRanges() { return zcr != null; }

[GitHub] [orc] dongjoon-hyun commented on issue #1583: Release Apache ORC 1.8.5

2023-09-08 Thread via GitHub
dongjoon-hyun commented on issue #1583: URL: https://github.com/apache/orc/issues/1583#issuecomment-1711819489 Thank you again, @wgtmac . Since this is done, let me close this issue too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [orc] dongjoon-hyun closed issue #1583: Release Apache ORC 1.8.5

2023-09-08 Thread via GitHub
dongjoon-hyun closed issue #1583: Release Apache ORC 1.8.5 URL: https://github.com/apache/orc/issues/1583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [orc] dongjoon-hyun closed pull request #1606: ORC-1499: Add MacOS 13 and 14 to `building.md`

2023-08-30 Thread via GitHub
dongjoon-hyun closed pull request #1606: ORC-1499: Add MacOS 13 and 14 to `building.md` URL: https://github.com/apache/orc/pull/1606 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [orc] dongjoon-hyun commented on pull request #1594: ORC-1489: Assign a writer id to CUDF

2023-08-30 Thread via GitHub
dongjoon-hyun commented on PR #1594: URL: https://github.com/apache/orc/pull/1594#issuecomment-1699505990 Apache ORC website is updated first. - https://orc.apache.org/specification/ORCv1/ ![Screenshot 2023-08-30 at 9 35 00 

[GitHub] [orc] dongjoon-hyun opened a new pull request, #1606: ORC-1499: Add MacOS 13 and 14 to `building.md`

2023-08-30 Thread via GitHub
dongjoon-hyun opened a new pull request, #1606: URL: https://github.com/apache/orc/pull/1606 ### What changes were proposed in this pull request? This PR aims to add MacOS 13 and 14 to `building.md`. ### Why are the changes needed? - #1511 added MacOS 13 test coverage .

[GitHub] [orc] wgtmac closed pull request #1607: ORC-1315: [C++] Fix byte to integer conversions fail on platforms with unsigned char type

2023-08-31 Thread via GitHub
wgtmac closed pull request #1607: ORC-1315: [C++] Fix byte to integer conversions fail on platforms with unsigned char type URL: https://github.com/apache/orc/pull/1607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [orc] wgtmac commented on issue #1583: Release Apache ORC 1.8.5

2023-08-30 Thread via GitHub
wgtmac commented on issue #1583: URL: https://github.com/apache/orc/issues/1583#issuecomment-1700235186 > BTW, for me, it looks like a dev environment issue exists a the same time. > > ``` > ==> logs/ubuntu20_jdk=11_cc=clang-test.log <== > /root/orc/c++/src/ByteRLE.cc:469:36:

[GitHub] [orc] dongjoon-hyun commented on pull request #1594: ORC-1489: Assign a writer id to CUDF

2023-08-29 Thread via GitHub
dongjoon-hyun commented on PR #1594: URL: https://github.com/apache/orc/pull/1594#issuecomment-1698436662 Merged to main/1.9/1.8. Thank you, @guiyanakuang and @wgtmac . BTW, to @guiyanakuang , I used `dev/merge_orc_pr.py` to merge to `main/1.9/1.8` branches and update JIRA together.

[GitHub] [orc] dongjoon-hyun commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-29 Thread via GitHub
dongjoon-hyun commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309605929 ## java/core/src/java/org/apache/orc/DataReader.java: ## @@ -54,9 +54,18 @@ BufferChunkList readFileData(BufferChunkList range, /** * Releases buffers created

[GitHub] [orc] dongjoon-hyun closed pull request #1594: ORC-1489: Assign a writer id to CUDF

2023-08-29 Thread via GitHub
dongjoon-hyun closed pull request #1594: ORC-1489: Assign a writer id to CUDF URL: https://github.com/apache/orc/pull/1594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [orc] dongjoon-hyun closed issue #1603: ORC-1489: Assign a writer id to CUDF

2023-08-29 Thread via GitHub
dongjoon-hyun closed issue #1603: ORC-1489: Assign a writer id to CUDF URL: https://github.com/apache/orc/issues/1603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [orc] dongjoon-hyun commented on pull request #1594: ORC-1489: Assign a writer id to CUDF

2023-08-29 Thread via GitHub
dongjoon-hyun commented on PR #1594: URL: https://github.com/apache/orc/pull/1594#issuecomment-1698439318 Additionally, GitHub Milestone feature doesn't allow multiple versions on a single PR. So, I created a clone of this for milestone 1.9.2 separately. It's #1603. -- This is an

[GitHub] [orc] dongjoon-hyun commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-29 Thread via GitHub
dongjoon-hyun commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309613871 ## java/core/src/test/org/apache/orc/impl/MockDFSDataInputStream.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [orc] dongjoon-hyun commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-29 Thread via GitHub
dongjoon-hyun commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309613871 ## java/core/src/test/org/apache/orc/impl/MockDFSDataInputStream.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

[GitHub] [orc] dongjoon-hyun commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-29 Thread via GitHub
dongjoon-hyun commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309615492 ## java/core/src/test/org/apache/orc/impl/TestRecordReaderUtils.java: ## @@ -140,6 +146,44 @@ public void testExtraBytesReadWithinThreshold() {

[GitHub] [orc] guiyanakuang commented on a diff in pull request #1594: ORC-1489: Assign a writer id to CUDF

2023-08-29 Thread via GitHub
guiyanakuang commented on code in PR #1594: URL: https://github.com/apache/orc/pull/1594#discussion_r1309521817 ## site/specification/ORCv1.md: ## @@ -136,6 +136,7 @@ message Footer { // 2 = Presto // 3 = Scritchley Go from https://github.com/scritchley/orc // 4 = Trino +

[GitHub] [orc] guiyanakuang commented on a diff in pull request #1594: ORC-1489: Assign a writer id to CUDF

2023-08-29 Thread via GitHub
guiyanakuang commented on code in PR #1594: URL: https://github.com/apache/orc/pull/1594#discussion_r1309521817 ## site/specification/ORCv1.md: ## @@ -136,6 +136,7 @@ message Footer { // 2 = Presto // 3 = Scritchley Go from https://github.com/scritchley/orc // 4 = Trino +

[GitHub] [orc] zxf216 opened a new issue, #1615: Has the ORC file encryption and decryption functionality been implemented?> Hi @dongjoon-hyun Do we have jira to track the work?

2023-09-13 Thread via GitHub
zxf216 opened a new issue, #1615: URL: https://github.com/apache/orc/issues/1615 > Hi @dongjoon-hyun Do we have jira to track the work? @deshanxiao Not yet. @coderex2522 is working on it already. Will create JIRAs later. _Originally posted by @wgtmac in

[GitHub] [orc] dongjoon-hyun closed pull request #1614: ORC-1503: Updated `README.md` with Maven version 3.9.4

2023-09-14 Thread via GitHub
dongjoon-hyun closed pull request #1614: ORC-1503: Updated `README.md` with Maven version 3.9.4 URL: https://github.com/apache/orc/pull/1614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [orc] dependabot[bot] opened a new pull request, #1608: Bump slf4j.version from 2.0.7 to 2.0.9 in /java

2023-09-04 Thread via GitHub
dependabot[bot] opened a new pull request, #1608: URL: https://github.com/apache/orc/pull/1608 Bumps `slf4j.version` from 2.0.7 to 2.0.9. Updates `org.slf4j:slf4j-api` from 2.0.7 to 2.0.9 Updates `org.slf4j:slf4j-simple` from 2.0.7 to 2.0.9 Dependabot will resolve any

[GitHub] [orc] kumonlin opened a new pull request, #1609: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
kumonlin opened a new pull request, #1609: URL: https://github.com/apache/orc/pull/1609 In the native orc scenario, the C++ orc reader does not support the partition field containing English special characters. As a result, the SQL job fails to be executed and no execution result is

[GitHub] [orc] dongjoon-hyun closed pull request #1604: ORC-1496: Use iterator to suggest backporting branches

2023-08-29 Thread via GitHub
dongjoon-hyun closed pull request #1604: ORC-1496: Use iterator to suggest backporting branches URL: https://github.com/apache/orc/pull/1604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [orc] dongjoon-hyun opened a new pull request, #1604: ORC-1496: Use iterator to suggest backporting branches

2023-08-29 Thread via GitHub
dongjoon-hyun opened a new pull request, #1604: URL: https://github.com/apache/orc/pull/1604 ### What changes were proposed in this pull request? This PR aims to use iterator to suggest backporting branches. ### Why are the changes needed? Previously, only `branch-1.9`

[GitHub] [orc] dependabot[bot] commented on pull request #1602: ORC-1497: Bump `maven-enforcer-plugin` to 3.4.0

2023-08-29 Thread via GitHub
dependabot[bot] commented on PR #1602: URL: https://github.com/apache/orc/pull/1602#issuecomment-1698510350 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me

[GitHub] [orc] dongjoon-hyun opened a new pull request, #1605: ORC-1498: Add Debian 12 Docker test

2023-08-30 Thread via GitHub
dongjoon-hyun opened a new pull request, #1605: URL: https://github.com/apache/orc/pull/1605 ### What changes were proposed in this pull request? This PR aims to add `Debian 12` Docker test. ### Why are the changes needed? To improve the test coverage for the new Debian

[GitHub] [orc] dongjoon-hyun closed pull request #1602: ORC-1497: Bump `maven-enforcer-plugin` to 3.4.0

2023-08-29 Thread via GitHub
dongjoon-hyun closed pull request #1602: ORC-1497: Bump `maven-enforcer-plugin` to 3.4.0 URL: https://github.com/apache/orc/pull/1602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [orc] wgtmac commented on pull request #1609: [C++] The partition field does not support English special characters

2023-09-04 Thread via GitHub
wgtmac commented on PR #1609: URL: https://github.com/apache/orc/pull/1609#issuecomment-1705346269 Thanks for submitting a PR! BTW, could you please keep the PR template? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [orc] dongjoon-hyun commented on pull request #1605: ORC-1498: Add `Debian 12` Docker test

2023-08-30 Thread via GitHub
dongjoon-hyun commented on PR #1605: URL: https://github.com/apache/orc/pull/1605#issuecomment-1698610525 Since this is `Dockerfile` and documentation only change, I'll merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [orc] dongjoon-hyun closed pull request #1605: ORC-1498: Add `Debian 12` Docker test

2023-08-30 Thread via GitHub
dongjoon-hyun closed pull request #1605: ORC-1498: Add `Debian 12` Docker test URL: https://github.com/apache/orc/pull/1605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [orc] dongjoon-hyun commented on pull request #1605: ORC-1498: Add `Debian 12` Docker test

2023-08-30 Thread via GitHub
dongjoon-hyun commented on PR #1605: URL: https://github.com/apache/orc/pull/1605#issuecomment-1698627162 Merged to main/1.9. `branch-1.9` is also tested. ``` Run CPack packaging tool... CPack: Create package using TGZ CPack: Install projects CPack: - Run preinstall target

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-30 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309798762 ## java/core/src/java/org/apache/orc/DataReader.java: ## @@ -54,9 +54,18 @@ BufferChunkList readFileData(BufferChunkList range, /** * Releases buffers created by

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-30 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309805678 ## java/core/src/test/org/apache/orc/impl/MockDFSDataInputStream.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-30 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309798762 ## java/core/src/java/org/apache/orc/DataReader.java: ## @@ -54,9 +54,18 @@ BufferChunkList readFileData(BufferChunkList range, /** * Releases buffers created by

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-30 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309798762 ## java/core/src/java/org/apache/orc/DataReader.java: ## @@ -54,9 +54,18 @@ BufferChunkList readFileData(BufferChunkList range, /** * Releases buffers created by

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-30 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309812591 ## java/core/src/test/org/apache/orc/impl/TestRecordReaderUtils.java: ## @@ -140,6 +146,44 @@ public void testExtraBytesReadWithinThreshold() {

[GitHub] [orc] xbthink commented on a diff in pull request #1574: ORC-1473: Fix zero copy read bugs

2023-08-30 Thread via GitHub
xbthink commented on code in PR #1574: URL: https://github.com/apache/orc/pull/1574#discussion_r1309823185 ## java/core/src/test/org/apache/orc/impl/TestRecordReaderUtils.java: ## @@ -140,6 +146,44 @@ public void testExtraBytesReadWithinThreshold() {

[PR] Bump com.github.spotbugs:spotbugs from 4.7.3 to 4.8.0 in /java [orc]

2023-10-16 Thread via GitHub
dependabot[bot] opened a new pull request, #1637: URL: https://github.com/apache/orc/pull/1637 Bumps [com.github.spotbugs:spotbugs](https://github.com/spotbugs/spotbugs) from 4.7.3 to 4.8.0. Release notes Sourced from

[PR] Bump com.google.guava:guava from 32.1.2-jre to 32.1.3-jre in /java [orc]

2023-10-16 Thread via GitHub
dependabot[bot] opened a new pull request, #1638: URL: https://github.com/apache/orc/pull/1638 Bumps [com.google.guava:guava](https://github.com/google/guava) from 32.1.2-jre to 32.1.3-jre. Release notes Sourced from

Re: [PR] ORC-1356: [C++] Use Intel AVX-512 instructions to accelerate the Rle-bit-packing decode [orc]

2023-10-13 Thread via GitHub
wpleonardo commented on PR #1375: URL: https://github.com/apache/orc/pull/1375#issuecomment-1761397725 > @wpleonardo still find no improvement if just select int64 type columns. > > Q: `select

[PR] ORD-1520: Removed JDK 8 settings from pom [orc]

2023-10-16 Thread via GitHub
mystic-lama opened a new pull request, #1639: URL: https://github.com/apache/orc/pull/1639 ### What changes were proposed in this pull request? The PR proposes to remove jdk8 related parameters from pom, since we moved to JDK 17 as part of #1627 ### Why are the changes

Re: [PR] ORC-1524: Bump `maven-shade-plugin` to 3.5.1 [orc]

2023-10-30 Thread via GitHub
dependabot[bot] commented on PR #1625: URL: https://github.com/apache/orc/pull/1625#issuecomment-1786301515 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me

Re: [PR] ORC-1524: Bump `maven-shade-plugin` to 3.5.1 [orc]

2023-10-30 Thread via GitHub
dongjoon-hyun closed pull request #1625: ORC-1524: Bump `maven-shade-plugin` to 3.5.1 URL: https://github.com/apache/orc/pull/1625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
dongjoon-hyun commented on PR #1641: URL: https://github.com/apache/orc/pull/1641#issuecomment-1789087522 Thank you, @wgtmac ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
dongjoon-hyun commented on PR #1641: URL: https://github.com/apache/orc/pull/1641#issuecomment-1789118468 Unfortunately, it turns out that we need additional PR for branch-1.8 because `std::make_unique` is not available in `branch-1.8`. This PR is merged to `main` and `branch-1.9`.

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-01 Thread via GitHub
neopaf commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1789094901 Reading same code I feel that ``` orc.bloom.filter.columns=* ``` Might work, but it would enable bloom filter for ALL columns, which is too much. -- This is an

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
dongjoon-hyun commented on PR #1641: URL: https://github.com/apache/orc/pull/1641#issuecomment-1789123268 To @hoffermei , I'd like to assign ORC-1525 to you. Do you have Apache JIRA ID? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
wgtmac commented on PR #1641: URL: https://github.com/apache/orc/pull/1641#issuecomment-1789137387 > Unfortunately, it turns out that we need additional PR for branch-1.8 because `std::make_unique` is not available in `branch-1.8`. > > This PR is merged to `main` and `branch-1.9`.

Re: [I] ORC-1525: [C++] suppress present stream may cause read orc failed with bad read in RleDecoderV2::readByte [orc]

2023-11-01 Thread via GitHub
dongjoon-hyun closed issue #1640: ORC-1525: [C++] suppress present stream may cause read orc failed with bad read in RleDecoderV2::readByte URL: https://github.com/apache/orc/issues/1640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
dongjoon-hyun closed pull request #1641: ORC-1525: Fix bad read in `RleDecoderV2::readByte` URL: https://github.com/apache/orc/pull/1641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
wgtmac opened a new pull request, #1645: URL: https://github.com/apache/orc/pull/1645 ### What changes were proposed in this pull request? This PR aims to fix #1640 by resetting `BooleanRleEncoderImpl::current` and `BooleanRleEncoderImpl::bitsRemained` when suppress ###

Re: [PR] ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON [orc]

2023-11-01 Thread via GitHub
wgtmac commented on PR #1646: URL: https://github.com/apache/orc/pull/1646#issuecomment-1789952964 cc @ffacs @coderex2522 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON [orc]

2023-11-01 Thread via GitHub
wgtmac opened a new pull request, #1646: URL: https://github.com/apache/orc/pull/1646 ### What changes were proposed in this pull request? Pass IOCount to SCOPED_STOPWATCH when counting I/Os. ### Why are the changes needed? SCOPED_STOPWATCH macro was not correctly called when

Re: [PR] ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON [orc]

2023-11-01 Thread via GitHub
ffacs commented on PR #1646: URL: https://github.com/apache/orc/pull/1646#issuecomment-1790006847 +1,LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-01 Thread via GitHub
wgtmac commented on PR #1645: URL: https://github.com/apache/orc/pull/1645#issuecomment-1789944074 @hoffermei @dongjoon-hyun PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-01 Thread via GitHub
guiyanakuang commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1790017243 @neopaf, I think org.apache.orc.OrcUtils#includeColumns will work. You can run this test code. ```java @Test public void test() { TypeDescription typeDescription

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
guiyanakuang commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1790196924 You can check out this doc https://orc.apache.org/docs/types.html. Even if the table creation statement only has a data column, the top-level root implicitly creates a

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
guiyanakuang commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1790253172 @neopaf. I can’t find any relevant reading material either. At the moment it seems that looking at the source code is the most direct way to go.

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
neopaf commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1790207301 @guiyanakuang thanks for the doc link and root-structure explanation. A bit fog left in "array" part: > Lists always have a single child column for the element values

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
neopaf closed issue #1644: orc.bloom.filter.columns=? (structhttps://github.com/apache/orc/issues/1644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
neopaf commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1790172263 looks perfect. sorry for not testing this thoroughly myself first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
neopaf commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1790181950 @guiyanakuang dear Yiqun, please suggest some good reading material as to how array's are mapped to columns? I've very confused. "index number of id is 5" is a mystery to me as of

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-02 Thread via GitHub
dongjoon-hyun closed pull request #1645: ORC-1525: Fix bad read in `RleDecoderV2::readByte` URL: https://github.com/apache/orc/pull/1645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] ORC-1525: Fix bad read in `RleDecoderV2::readByte` [orc]

2023-11-02 Thread via GitHub
dongjoon-hyun commented on PR #1645: URL: https://github.com/apache/orc/pull/1645#issuecomment-1791546913 Merged to branch-1.8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] orc.bloom.filter.columns=? (struct

2023-11-02 Thread via GitHub
neopaf commented on issue #1644: URL: https://github.com/apache/orc/issues/1644#issuecomment-1791363254 @guiyanakuang came across this quick description: https://orc.apache.org/specification/ORCv0/ Looks like I had a total misconception inside my head that if we have an array the

Re: [PR] ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON [orc]

2023-11-02 Thread via GitHub
wgtmac closed pull request #1646: ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON URL: https://github.com/apache/orc/pull/1646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON [orc]

2023-11-02 Thread via GitHub
wgtmac commented on PR #1646: URL: https://github.com/apache/orc/pull/1646#issuecomment-1791917219 Thank you @ffacs and @dongjoon-hyun ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] ORC-1480: [C++] Fix build break w/ BUILD_CPP_ENABLE_METRICS=ON [orc]

2023-11-02 Thread via GitHub
wgtmac commented on PR #1646: URL: https://github.com/apache/orc/pull/1646#issuecomment-1791918483 I have also backported it into branch-1.9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] ORC-1432: Add MacOS 13 GitHub Action Job [orc]

2023-11-04 Thread via GitHub
dongjoon-hyun closed issue #1648: ORC-1432: Add MacOS 13 GitHub Action Job URL: https://github.com/apache/orc/issues/1648 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] ORC-1432: Add MacOS 13 GitHub Action Job [orc]

2023-11-04 Thread via GitHub
dongjoon-hyun commented on issue #1648: URL: https://github.com/apache/orc/issues/1648#issuecomment-1793619205 This is resolved via https://github.com/apache/orc/commit/db5ecfcbc75537fcf45aa76996090c7216e51085 -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] ORC-1499: Add MacOS 13 and 14 to `building.md` [orc]

2023-11-04 Thread via GitHub
dongjoon-hyun closed issue #1649: ORC-1499: Add MacOS 13 and 14 to `building.md` URL: https://github.com/apache/orc/issues/1649 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] ORC-1499: Add MacOS 13 and 14 to `building.md` [orc]

2023-11-04 Thread via GitHub
dongjoon-hyun commented on issue #1649: URL: https://github.com/apache/orc/issues/1649#issuecomment-1793622486 This is resolved via 72c9397e66b1a6f2a35129d7c14afbc7457e6f30 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] ORC-1482: Adaptation to read ORC files created by CUDF [orc]

2023-11-05 Thread via GitHub
dongjoon-hyun closed issue #1651: ORC-1482: Adaptation to read ORC files created by CUDF URL: https://github.com/apache/orc/issues/1651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] ORC-1482: Adaptation to read ORC files created by CUDF [orc]

2023-11-05 Thread via GitHub
dongjoon-hyun commented on issue #1651: URL: https://github.com/apache/orc/issues/1651#issuecomment-1793793090 This is resolved via b3016d779e79d71ab70a283fb292390a5ae37100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] ORC-1489: Assign a writer id to CUDF [orc]

2023-11-05 Thread via GitHub
dongjoon-hyun commented on issue #1652: URL: https://github.com/apache/orc/issues/1652#issuecomment-1793793259 This is resolved via 488184b33eb3d662e89ecc10f2c3e5fab6f34bf5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

  1   2   3   4   5   6   7   8   9   10   >