[jira] [Updated] (HUDI-7361) Fix a concurrency issue caused by rollbackFailedWrites

2024-01-30 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-7361: --- Description: {quote}CREATE TABLE tbl ( .. ) WITH ( 'connector' = 'hudi', 'path' = '/tblpath', 'table.type' = 'COPY

[jira] [Updated] (HUDI-7361) Fix a concurrency issue caused by rollbackFailedWrites

2024-01-30 Thread eric (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eric updated HUDI-7361: --- Attachment: jobmanager_log.txt taskmanager_log.txt > Fix a concurrency issue caused by rollbackFailedW

[jira] [Created] (HUDI-7361) Fix a concurrency issue caused by rollbackFailedWrites

2024-01-30 Thread eric (Jira)
eric created HUDI-7361: -- Summary: Fix a concurrency issue caused by rollbackFailedWrites Key: HUDI-7361 URL: https://issues.apache.org/jira/browse/HUDI-7361 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-7360) Incremental CDC Query after 0.14.1 upgrade giving Jackson class incompatibility exception

2024-01-30 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka updated HUDI-7360: Description: Github Issue - [https://github.com/apache/hudi/issues/10590] Reproducible code  ``` f

[jira] [Updated] (HUDI-7360) Incremental CDC Query after 0.14.1 upgrade giving Jackson class incompatibility exception

2024-01-30 Thread Aditya Goenka (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Goenka updated HUDI-7360: Summary: Incremental CDC Query after 0.14.1 upgrade giving Jackson class incompatibility exception

[jira] [Created] (HUDI-7360) Incremental CDC Query after 0.14.X upgrade giving Jackson class incompatibility exception

2024-01-30 Thread Aditya Goenka (Jira)
Aditya Goenka created HUDI-7360: --- Summary: Incremental CDC Query after 0.14.X upgrade giving Jackson class incompatibility exception Key: HUDI-7360 URL: https://issues.apache.org/jira/browse/HUDI-7360 P

[jira] [Commented] (HUDI-7320) hive-sync unexpectedly loads archived timeline

2024-01-30 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812499#comment-17812499 ] sivabalan narayanan commented on HUDI-7320: --- We did fix something on these lines

Re: [PR] added new videos for hudi oss site [hudi]

2024-01-30 Thread via GitHub
nfarah86 commented on PR #10563: URL: https://github.com/apache/hudi/pull/10563#issuecomment-1917696609 @bhasudha made the aws -> amazon changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [HUDI-7357] Introduce generic StorageConfiguration [hudi]

2024-01-30 Thread via GitHub
hudi-bot commented on PR #10586: URL: https://github.com/apache/hudi/pull/10586#issuecomment-1917684020 ## CI report: * e6a99b7319648fce943abc73b460239350ff18d3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7347][Stacked on HUDI-7335] Introduce SeekableDataInputStream for random access [hudi]

2024-01-30 Thread via GitHub
hudi-bot commented on PR #10575: URL: https://github.com/apache/hudi/pull/10575#issuecomment-1917510979 ## CI report: * 24d06d5c92ebb9ef98c4689365eabd1e197c7197 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7357] Introduce generic StorageConfiguration [hudi]

2024-01-30 Thread via GitHub
hudi-bot commented on PR #10586: URL: https://github.com/apache/hudi/pull/10586#issuecomment-1917495130 ## CI report: * e6a99b7319648fce943abc73b460239350ff18d3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7347][Stacked on HUDI-7335] Introduce SeekableDataInputStream for random access [hudi]

2024-01-30 Thread via GitHub
hudi-bot commented on PR #10575: URL: https://github.com/apache/hudi/pull/10575#issuecomment-1917494985 ## CI report: * 24d06d5c92ebb9ef98c4689365eabd1e197c7197 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7357] Introduce generic StorageConfiguration [hudi]

2024-01-30 Thread via GitHub
hudi-bot commented on PR #10586: URL: https://github.com/apache/hudi/pull/10586#issuecomment-1917480260 ## CI report: * e6a99b7319648fce943abc73b460239350ff18d3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7347][Stacked on HUDI-7335] Introduce SeekableDataInputStream for random access [hudi]

2024-01-30 Thread via GitHub
yihua commented on code in PR #10575: URL: https://github.com/apache/hudi/pull/10575#discussion_r1471603752 ## hudi-hadoop-common/src/main/java/org/apache/hudi/hadoop/fs/HadoopSeekableDataInputStream.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (A

(hudi) branch master updated: [HUDI-7343] Replace Path.SEPARATOR with HoodieLocation.SEPARATOR (#10570)

2024-01-30 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new a078242b19d [HUDI-7343] Replace Path.SEPARATOR with

Re: [PR] [HUDI-7343] Replace Path.SEPARATOR with HoodieLocation.SEPARATOR [hudi]

2024-01-30 Thread via GitHub
yihua merged PR #10570: URL: https://github.com/apache/hudi/pull/10570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-7343] Replace Path.SEPARATOR with HoodieLocation.SEPARATOR [hudi]

2024-01-30 Thread via GitHub
yihua commented on PR #10570: URL: https://github.com/apache/hudi/pull/10570#issuecomment-1917304625 > I didn't check all the usages of the `Path.SEPARATOR`, the change looks straight-forward so I approved it. Yes, `Path.SEPARATOR` usages are all replaced. -- This is an automated m

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-30 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1471439782 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/ObjectInspectorCache.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-30 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1471420556 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java: ## @@ -91,9 +94,42 @@ private void initAvroInputFormat() { } } + private

Re: [PR] [HUDI-7045] Create parquet readers inside the reader context and implement schema.on.read in the filegroup reader in spark [hudi]

2024-01-30 Thread via GitHub
jonvex commented on PR #10278: URL: https://github.com/apache/hudi/pull/10278#issuecomment-1917113112 Azure CI all passing @yihua -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] added new videos for hudi oss site [hudi]

2024-01-30 Thread via GitHub
bhasudha commented on code in PR #10563: URL: https://github.com/apache/hudi/pull/10563#discussion_r1471267966 ## website/videoBlog/2023-10-14-Accelerating-Data-Processing-Leveraging-Apache-Hudi-with-DynamoDB-for-Faster-Commit-Time-Retrieval.md: ## @@ -8,7 +8,7 @@ image: /asset

Re: [PR] added new videos for hudi oss site [hudi]

2024-01-30 Thread via GitHub
bhasudha commented on code in PR #10563: URL: https://github.com/apache/hudi/pull/10563#discussion_r1471267410 ## website/videoBlog/2023-08-06-Easy_Step_by_Step_Guide_for_Beginner_Setup_AWS_Transfer_Family_SFTP_with_S3.md: ## @@ -11,7 +11,7 @@ tags: - third-party data - sftp

Re: [PR] added new videos for hudi oss site [hudi]

2024-01-30 Thread via GitHub
bhasudha commented on code in PR #10563: URL: https://github.com/apache/hudi/pull/10563#discussion_r1471266750 ## website/videoBlog/2023-08-03-Powering_EventDriven_Workloads_with_Hudi_Read_Stream_AWS_Glue_Streaming_JOBS.md: ## @@ -14,6 +14,6 @@ tags: - streaming - near real-ti

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-30 Thread via GitHub
xiarixiaoyao commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1471121556 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/ObjectInspectorCache.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[I] [hudi bucket prune] [hudi]

2024-01-30 Thread via GitHub
lookingUpAtTheSky opened a new issue, #10589: URL: https://github.com/apache/hudi/issues/10589 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at

Re: [I] [SUPPORT] Querying Hudi tables with Spark+Velox(C++), ObjectSizeCalculator.getObjectSize hangs causing about a 50-second delay in queries [hudi]

2024-01-30 Thread via GitHub
majian1998 commented on issue #10580: URL: https://github.com/apache/hudi/issues/10580#issuecomment-1916596628 @ad1happy2go I understand that the issue started when the PR [HUDI-4687] introduced the use of jol to estimate object size. -- This is an automated message from the Apache Git S

Re: [I] [SUPPORT] Querying Hudi tables with Spark+Velox(C++), ObjectSizeCalculator.getObjectSize hangs causing about a 50-second delay in queries [hudi]

2024-01-30 Thread via GitHub
ad1happy2go commented on issue #10580: URL: https://github.com/apache/hudi/issues/10580#issuecomment-1916559418 @majian1998 Is this issue occurring after 0.14.0 upgrade or it was happening with older Hudi version too? -- This is an automated message from the Apache Git Service. To res

Re: [I] [SUPPORT] Can't delete key (row) for all commits in HUDI Table (history)? [hudi]

2024-01-30 Thread via GitHub
ad1happy2go commented on issue #10581: URL: https://github.com/apache/hudi/issues/10581#issuecomment-1916557526 @jens4doc Dont think there is a way to achieve that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Hudi 6868 - Support extracting passwords from credential store for Hive Sync [hudi]

2024-01-30 Thread via GitHub
hudi-bot commented on PR #10577: URL: https://github.com/apache/hudi/pull/10577#issuecomment-1916344140 ## CI report: * 40cbc324442334d3e1313f995c8ae9feed7d0db7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [I] [Support] An error occurred while calling o1748.load.\n: java.io.FileNotFoundException [hudi]

2024-01-30 Thread via GitHub
gsudhanshu commented on issue #10503: URL: https://github.com/apache/hudi/issues/10503#issuecomment-1916285606 @ad1happy2go thanks for your inputs. I had made changes in path and removed unneccary keys. But still facing the same issue of Filenotfound exception. It seems like I