[GitHub] [hudi] bhasudha opened a new pull request, #6477: [DOCS] Change community sync schedule image

2022-08-22 Thread GitBox
bhasudha opened a new pull request, #6477: URL: https://github.com/apache/hudi/pull/6477 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

[GitHub] [hudi] hudi-bot commented on pull request #6467: [HUDI-4686] Flip option 'write.ignore.failed' to default false

2022-08-22 Thread GitBox
hudi-bot commented on PR #6467: URL: https://github.com/apache/hudi/pull/6467#issuecomment-1223562209 ## CI report: * 23b77552e300ca697b142ebe687cf2a8b4452bfa Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6000: [HUDI-4340] fix not parsable text DateTimeParseException in HoodieInstantTimeGenerator.parseDateFromInstantTime

2022-08-22 Thread GitBox
hudi-bot commented on PR #6000: URL: https://github.com/apache/hudi/pull/6000#issuecomment-1223561595 ## CI report: * b54e1a1397b1294cc4dc6e28bdfea7fb4ccaceab Azure:

[GitHub] [hudi] xushiyan commented on pull request #6256: [RFC-51][HUDI-3478] Update RFC: CDC support

2022-08-22 Thread GitBox
xushiyan commented on PR #6256: URL: https://github.com/apache/hudi/pull/6256#issuecomment-1223535084 > @xushiyan - Do you want to take a final stab at the RFC incorporating all the changes? I believe we have consensus on what needs to be done. Please correct me if I am wrong. cc

[GitHub] [hudi] YannByron commented on pull request #5885: [HUDI-3478] Support CDC for Spark in Hudi

2022-08-22 Thread GitBox
YannByron commented on PR #5885: URL: https://github.com/apache/hudi/pull/5885#issuecomment-1223521172 Reopen: https://github.com/apache/hudi/pull/6476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] hudi-bot commented on pull request #6000: [HUDI-4340] fix not parsable text DateTimeParseException in HoodieInstantTimeGenerator.parseDateFromInstantTime

2022-08-22 Thread GitBox
hudi-bot commented on PR #6000: URL: https://github.com/apache/hudi/pull/6000#issuecomment-1223518699 ## CI report: * 3782118990698553ac6121b49641e79e01407353 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6000: [HUDI-4340] fix not parsable text DateTimeParseException in HoodieInstantTimeGenerator.parseDateFromInstantTime

2022-08-22 Thread GitBox
hudi-bot commented on PR #6000: URL: https://github.com/apache/hudi/pull/6000#issuecomment-1223515434 ## CI report: * 3782118990698553ac6121b49641e79e01407353 Azure:

[GitHub] [hudi] hehuiyuan commented on pull request #6392: [HUDI-4618][common]Separate log word for CommitUitls class

2022-08-22 Thread GitBox
hehuiyuan commented on PR #6392: URL: https://github.com/apache/hudi/pull/6392#issuecomment-1223493994 @danny0405 hi, take a look when you have time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Assigned] (HUDI-4677) Snapshot view management

2022-08-22 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4677: --- Assignee: Jian Feng > Snapshot view management > > > Key:

[GitHub] [hudi] dwshmilyss closed issue #6470: [SUPPORT]SHOW PARTITIONS is not allowed on hudi table since its partition metadata is not stored in the Hive metastore

2022-08-22 Thread GitBox
dwshmilyss closed issue #6470: [SUPPORT]SHOW PARTITIONS is not allowed on hudi table since its partition metadata is not stored in the Hive metastore URL: https://github.com/apache/hudi/issues/6470 -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hudi] awsUser123 opened a new issue, #6475: [SUPPORT]

2022-08-22 Thread GitBox
awsUser123 opened a new issue, #6475: URL: https://github.com/apache/hudi/issues/6475 Hey guys, I am trying to implement reading from kinesis data streams and storing it into an s3 bucket using Hudi. I was able to add the data into s3 by referring and running the following code-

[GitHub] [hudi] LinMingQiang commented on issue #5330: [SUPPORT] [BUG] Duplicate fileID ??? from bucket ?? of partition found during the BucketStreamWriteFunction index bootstrap.

2022-08-22 Thread GitBox
LinMingQiang commented on issue #5330: URL: https://github.com/apache/hudi/issues/5330#issuecomment-1223475113 see https://github.com/apache/hudi/pull/5763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] hudi-bot commented on pull request #6467: [HUDI-4686] Flip option 'write.ignore.failed' to default false

2022-08-22 Thread GitBox
hudi-bot commented on PR #6467: URL: https://github.com/apache/hudi/pull/6467#issuecomment-1223473325 ## CI report: * e9b3607a806759544aa333ac256cdf95e5434ce3 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6467: [HUDI-4686] Flip option 'write.ignore.failed' to default false

2022-08-22 Thread GitBox
hudi-bot commented on PR #6467: URL: https://github.com/apache/hudi/pull/6467#issuecomment-1223470017 ## CI report: * e9b3607a806759544aa333ac256cdf95e5434ce3 Azure:

[GitHub] [hudi] TengHuo commented on a diff in pull request #6000: [HUDI-4340] fix not parsable text DateTimeParseException in HoodieInstantTimeGenerator.parseDateFromInstantTime

2022-08-22 Thread GitBox
TengHuo commented on code in PR #6000: URL: https://github.com/apache/hudi/pull/6000#discussion_r952099366 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java: ## @@ -80,6 +80,15 @@ public class HoodieActiveTimeline extends

[jira] [Resolved] (HUDI-4676) infer cleaner policy when write concurrency mode is OCC

2022-08-22 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng resolved HUDI-4676. - > infer cleaner policy when write concurrency mode is OCC >

[jira] [Assigned] (HUDI-4676) infer cleaner policy when write concurrency mode is OCC

2022-08-22 Thread Jian Feng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Feng reassigned HUDI-4676: --- Assignee: Jian Feng > infer cleaner policy when write concurrency mode is OCC >

[GitHub] [hudi] dwshmilyss commented on issue #6470: [SUPPORT]SHOW PARTITIONS is not allowed on hudi table since its partition metadata is not stored in the Hive metastore

2022-08-22 Thread GitBox
dwshmilyss commented on issue #6470: URL: https://github.com/apache/hudi/issues/6470#issuecomment-1223449366 @Zouxxyy thanks for your advise,I found that this problem was caused by the conflict between SPARK3.2 and HUDI 0.11.1. In HUDI 0.11.1, HiveSyncTool. GetSparkTableProperties () the

[jira] [Commented] (HUDI-4384) Hive style partition not work and record key loss prefix using ComplexKey in bulk_insert

2022-08-22 Thread Teng Huo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583296#comment-17583296 ] Teng Huo commented on HUDI-4384: Got it, np. Thanks [~xushiyan] > Hive style partition not work and

[GitHub] [hudi] danny0405 commented on a diff in pull request #6000: [HUDI-4340] fix not parsable text DateTimeParseException in HoodieInstantTimeGenerator.parseDateFromInstantTime

2022-08-22 Thread GitBox
danny0405 commented on code in PR #6000: URL: https://github.com/apache/hudi/pull/6000#discussion_r952087908 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java: ## @@ -80,6 +80,15 @@ public class HoodieActiveTimeline extends

[GitHub] [hudi] danny0405 commented on pull request #6456: [HUDI-4674]Change the default value of inputFormat for the MOR table

2022-08-22 Thread GitBox
danny0405 commented on PR #6456: URL: https://github.com/apache/hudi/pull/6456#issuecomment-1223440395 @alexeykudinkin Can you help take a look here ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] danny0405 commented on pull request #6456: [HUDI-4674]Change the default value of inputFormat for the MOR table

2022-08-22 Thread GitBox
danny0405 commented on PR #6456: URL: https://github.com/apache/hudi/pull/6456#issuecomment-1223438615 > Sparksql.then we'll see the table inputFormat is HoodieParquetRealtimeInputFormat Thanks, we may need to figure out why the Spark sql uses `HoodieParquetRealtimeInputFormat` as

[jira] [Resolved] (HUDI-4683) Fix to use enum class value for default value in flinkoptions

2022-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-4683. -- > Fix to use enum class value for default value in flinkoptions >

[jira] [Updated] (HUDI-4683) Use enum class value for default value in flink options

2022-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4683: - Summary: Use enum class value for default value in flink options (was: Fix to use enum class value for

[jira] [Updated] (HUDI-4683) Fix to use enum class value for default value in flinkoptions

2022-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4683: - Fix Version/s: 0.12.1 > Fix to use enum class value for default value in flinkoptions >

[jira] [Commented] (HUDI-4683) Fix to use enum class value for default value in flinkoptions

2022-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583294#comment-17583294 ] Danny Chen commented on HUDI-4683: -- Fixed via master branch: c677333f26aaa4dc880a04b7532929b68bd978ed >

[hudi] branch master updated (4966978a55 -> c677333f26)

2022-08-22 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 4966978a55 [HUDI-4676] infer cleaner policy when write concurrency mode is OCC (#6459) add c677333f26

[GitHub] [hudi] danny0405 merged pull request #6453: [HUDI-4683] Use enum class value for default value in flink options

2022-08-22 Thread GitBox
danny0405 merged PR #6453: URL: https://github.com/apache/hudi/pull/6453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan commented on pull request #4665: [HUDI-2733] Add support for Thrift sync

2022-08-22 Thread GitBox
xushiyan commented on PR #4665: URL: https://github.com/apache/hudi/pull/4665#issuecomment-1223435214 @stym06 any chance you can rebase and update this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (HUDI-4261) OOM in bulk-insert when using "NONE" sort-mode for table w/ large # of partitions

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4261: - Sprint: 2022/08/22 > OOM in bulk-insert when using "NONE" sort-mode for table w/ large # of > partitions

[GitHub] [hudi] hudi-bot commented on pull request #6170: [HUDI-4441] Log4j2 configuration fixes and removal of log4j1 dependencies

2022-08-22 Thread GitBox
hudi-bot commented on PR #6170: URL: https://github.com/apache/hudi/pull/6170#issuecomment-1223422846 ## CI report: * 520b1b54c37ce6378047a55f650e309d6feb89d1 Azure:

[jira] [Closed] (HUDI-3806) Improve HoodieBloomIndex using bloom_filter and col_stats in MDT

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3806. Resolution: Duplicate > Improve HoodieBloomIndex using bloom_filter and col_stats in MDT >

[jira] [Updated] (HUDI-4585) Optimize query performance on Presto Hudi connector

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4585: - Story Points: 10 > Optimize query performance on Presto Hudi connector >

[jira] [Updated] (HUDI-4586) Address S3 timeouts in Bloom Index with metadata table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4586: - Story Points: 5 > Address S3 timeouts in Bloom Index with metadata table >

[jira] [Updated] (HUDI-3806) Improve HoodieBloomIndex using bloom_filter and col_stats in MDT

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3806: - Story Points: 0 (was: 4) > Improve HoodieBloomIndex using bloom_filter and col_stats in MDT >

[jira] [Updated] (HUDI-4586) Address S3 timeouts in Bloom Index with metadata table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4586: - Status: Patch Available (was: In Progress) > Address S3 timeouts in Bloom Index with metadata table >

[jira] [Assigned] (HUDI-1369) Bootstrap datasource jobs from hanging via spark-submit

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-1369: Assignee: Ethan Guo (was: Wenning Ding) > Bootstrap datasource jobs from hanging via spark-submit

[jira] [Updated] (HUDI-4125) Add IT (Azure CI) around bootstrapped Hudi table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4125: - Summary: Add IT (Azure CI) around bootstrapped Hudi table (was: Add integration tests around

[jira] [Commented] (HUDI-3495) Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to empty results even if key exists

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583290#comment-17583290 ] Raymond Xu commented on HUDI-3495: -- [~guoyihua]: to be verified before closing > Reading keys in

[jira] [Updated] (HUDI-3495) Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead to empty results even if key exists

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3495: - Sprint: (was: 2022/09/19) > Reading keys in parallel from HoodieMetadataMergedLogRecordReader may lead

[jira] [Updated] (HUDI-3777) Optimize column stats storage

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3777: - Sprint: (was: 2022/09/19) > Optimize column stats storage > - > >

[jira] [Updated] (HUDI-3777) Optimize column stats storage

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3777: - Description: Avoid storing filename of each record in the colstats partition. As of now, we store

[jira] [Updated] (HUDI-4669) Incorrect protoc executable in kafka-connect fails build on Mac M1

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4669: - Sprint: 2022/09/19 > Incorrect protoc executable in kafka-connect fails build on Mac M1 >

[hudi] branch master updated (3adb571531 -> 4966978a55)

2022-08-22 Thread forwardxu
This is an automated email from the ASF dual-hosted git repository. forwardxu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 3adb571531 [HUDI-4678] Claim RFC-61 for Snapshot view management (#6461) add 4966978a55 [HUDI-4676] infer

[GitHub] [hudi] XuQianJin-Stars merged pull request #6459: [HUDI-4676] infer cleaner policy when write concurrency mode is OCC

2022-08-22 Thread GitBox
XuQianJin-Stars merged PR #6459: URL: https://github.com/apache/hudi/pull/6459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HUDI-3519) Make sure every public Hudi Client Method invokes necessary prologue

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3519: - Component/s: code-quality > Make sure every public Hudi Client Method invokes necessary prologue >

[jira] [Updated] (HUDI-3519) Make sure every public Hudi Client Method invokes necessary prologue

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3519: - Priority: Major (was: Blocker) > Make sure every public Hudi Client Method invokes necessary prologue >

[jira] [Updated] (HUDI-3519) Make sure every public Hudi Client Method invokes necessary prologue

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3519: - Sprint: (was: 2022/09/19) > Make sure every public Hudi Client Method invokes necessary prologue >

[jira] [Updated] (HUDI-3301) MergedLogRecordReader inline reading should be stateless and thread safe

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3301: - Sprint: (was: 2022/09/19) > MergedLogRecordReader inline reading should be stateless and thread safe >

[jira] [Updated] (HUDI-3301) MergedLogRecordReader inline reading should be stateless and thread safe

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3301: - Priority: Major (was: Blocker) > MergedLogRecordReader inline reading should be stateless and thread

[jira] [Commented] (HUDI-3300) Timeline server FSViewManager should avoid point lookup for metadata file partition

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583284#comment-17583284 ] Raymond Xu commented on HUDI-3300: -- [~guoyihua] : can be closed after verification > Timeline server

[jira] [Updated] (HUDI-3300) Timeline server FSViewManager should avoid point lookup for metadata file partition

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3300: - Sprint: (was: 2022/09/19) > Timeline server FSViewManager should avoid point lookup for metadata file

[jira] [Updated] (HUDI-3300) Timeline server FSViewManager should avoid point lookup for metadata file partition

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3300: - Story Points: 0 (was: 2) > Timeline server FSViewManager should avoid point lookup for metadata file >

[jira] [Updated] (HUDI-3300) Timeline server FSViewManager should avoid point lookup for metadata file partition

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3300: - Priority: Major (was: Blocker) > Timeline server FSViewManager should avoid point lookup for metadata

[jira] [Updated] (HUDI-3453) Metadata table throws NPE when scheduling compaction plan

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3453: - Sprint: (was: 2022/08/22) > Metadata table throws NPE when scheduling compaction plan >

[jira] [Closed] (HUDI-1461) Bulk insert v2 creates additional small files

2022-08-22 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin closed HUDI-1461. - Resolution: Duplicate > Bulk insert v2 creates additional small files >

[jira] [Updated] (HUDI-3636) Clustering fails due to marker creation failure

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3636: - Reviewers: sivabalan narayanan > Clustering fails due to marker creation failure >

[jira] [Updated] (HUDI-4637) Release thread in RateLimiter is not terminated

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4637: - Reviewers: sivabalan narayanan > Release thread in RateLimiter is not terminated >

[jira] [Updated] (HUDI-4326) Hudi spark datasource error after migrate from 0.8 to 0.11

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4326: - Reviewers: Raymond Xu > Hudi spark datasource error after migrate from 0.8 to 0.11 >

[jira] [Updated] (HUDI-4635) Update roadmap page based on H2 2022 plan

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4635: - Reviewers: Raymond Xu > Update roadmap page based on H2 2022 plan >

[jira] [Assigned] (HUDI-4327) TestHoodieDeltaStreamer#testCleanerDeleteReplacedDataWithArchive is flaky

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4327: Assignee: sivabalan narayanan > TestHoodieDeltaStreamer#testCleanerDeleteReplacedDataWithArchive

[jira] [Assigned] (HUDI-4695) Flaky: TestInlineCompaction.testCompactionRetryOnFailureBasedOnTime:308 expected: <4> but was: <5>

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4695: Assignee: sivabalan narayanan > Flaky:

[jira] [Assigned] (HUDI-4696) Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4696: Assignee: Raymond Xu > Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer >

[jira] [Updated] (HUDI-4695) Flaky: TestInlineCompaction.testCompactionRetryOnFailureBasedOnTime:308 expected: <4> but was: <5>

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4695: - Description:

[jira] [Updated] (HUDI-4438) Fix flaky TestCopyOnWriteActionExecutor.testPartitionMetafileFormat test

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4438: - Sprint: (was: 2022/09/05) > Fix flaky TestCopyOnWriteActionExecutor.testPartitionMetafileFormat test >

[GitHub] [hudi] gudladona commented on issue #6474: [SUPPORT] Hudi Deltastreamer fails to acquire lock with DynamoDB Lock Provider.

2022-08-22 Thread GitBox
gudladona commented on issue #6474: URL: https://github.com/apache/hudi/issues/6474#issuecomment-1223386268 This seems to be more comprehensive sequence than the above ``` cat hudi-logs.txt | grep -E 'TransactionManager|DynamoDBBasedLockProvider'

[GitHub] [hudi] hudi-bot commented on pull request #6450: [HUDI-4665] Flipping default for "ignore failed batch" config in streaming sink to false

2022-08-22 Thread GitBox
hudi-bot commented on PR #6450: URL: https://github.com/apache/hudi/pull/6450#issuecomment-1223386217 ## CI report: * 3bd700dea82006f1d3081c3eee7ab1b430728911 Azure:

[jira] [Updated] (HUDI-3054) Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3054: - Sprint: (was: 2022/08/22) > Fix flaky TestHoodieClientMultiWriter. testHoodieClientBasicMultiWriter >

[jira] [Created] (HUDI-4696) Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer

2022-08-22 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-4696: Summary: Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer Key: HUDI-4696 URL: https://issues.apache.org/jira/browse/HUDI-4696 Project: Apache Hudi

[jira] [Updated] (HUDI-4696) Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4696: - Description:

[jira] [Updated] (HUDI-2528) Flaky test: MERGE_ON_READ testTableOperationsWithRestore

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2528: - Sprint: (was: 2022/08/22) > Flaky test: MERGE_ON_READ testTableOperationsWithRestore >

[jira] [Updated] (HUDI-4695) Flaky: TestInlineCompaction.testCompactionRetryOnFailureBasedOnTime:308 expected: <4> but was: <5>

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4695: - Story Points: 3 > Flaky: TestInlineCompaction.testCompactionRetryOnFailureBasedOnTime:308 > expected:

[jira] [Updated] (HUDI-4696) Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4696: - Story Points: 3 > Flaky: TestHoodieCombineHiveInputFormat.setUpClass:86 » NullPointer >

[GitHub] [hudi] hudi-bot commented on pull request #6170: [HUDI-4441] Log4j2 configuration fixes and removal of log4j1 dependencies

2022-08-22 Thread GitBox
hudi-bot commented on PR #6170: URL: https://github.com/apache/hudi/pull/6170#issuecomment-1223382898 ## CI report: * 520b1b54c37ce6378047a55f650e309d6feb89d1 Azure:

[jira] [Created] (HUDI-4695) Flaky: TestInlineCompaction.testCompactionRetryOnFailureBasedOnTime:308 expected: <4> but was: <5>

2022-08-22 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-4695: Summary: Flaky: TestInlineCompaction.testCompactionRetryOnFailureBasedOnTime:308 expected: <4> but was: <5> Key: HUDI-4695 URL: https://issues.apache.org/jira/browse/HUDI-4695

[GitHub] [hudi] gudladona commented on issue #6474: [SUPPORT] Hudi Deltastreamer fails to acquire lock with DynamoDB Lock Provider.

2022-08-22 Thread GitBox
gudladona commented on issue #6474: URL: https://github.com/apache/hudi/issues/6474#issuecomment-1223380423 Yes, it seems like the transaction that started at @ 02:06:31 20220822020402958__deltacommit__INFLIGHT is on the metadata table. Also, it appears like this was held for 25 minutes,

[GitHub] [hudi] hudi-bot commented on pull request #6170: [HUDI-4441] Log4j2 configuration fixes and removal of log4j1 dependencies

2022-08-22 Thread GitBox
hudi-bot commented on PR #6170: URL: https://github.com/apache/hudi/pull/6170#issuecomment-1223379502 ## CI report: * 520b1b54c37ce6378047a55f650e309d6feb89d1 Azure:

[GitHub] [hudi] yihua commented on a diff in pull request #6467: [HUDI-4686] Flip option 'write.ignore.failed' to default false

2022-08-22 Thread GitBox
yihua commented on code in PR #6467: URL: https://github.com/apache/hudi/pull/6467#discussion_r952040628 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java: ## @@ -327,9 +327,9 @@ private FlinkOptions() { public static final

[GitHub] [hudi] nsivabalan commented on issue #6474: [SUPPORT] Hudi Deltastreamer fails to acquire lock with DynamoDB Lock Provider.

2022-08-22 Thread GitBox
nsivabalan commented on issue #6474: URL: https://github.com/apache/hudi/issues/6474#issuecomment-1223377849 Note to self: excerpt from the logs which of interest to us ``` 22/08/22 02:06:31 INFO org.apache.hudi.client.transaction.TransactionManager: Transaction starting for

[GitHub] [hudi] hudi-bot commented on pull request #6432: [HUDI-4586] Improve metadata fetching in bloom index

2022-08-22 Thread GitBox
hudi-bot commented on PR #6432: URL: https://github.com/apache/hudi/pull/6432#issuecomment-1223375990 ## CI report: * ed15f57dc58b2e9142dd33a0ecd078bf4c236afc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6170: [HUDI-4441] Log4j2 configuration fixes and removal of log4j1 dependencies

2022-08-22 Thread GitBox
hudi-bot commented on PR #6170: URL: https://github.com/apache/hudi/pull/6170#issuecomment-1223375610 ## CI report: * 520b1b54c37ce6378047a55f650e309d6feb89d1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] nsivabalan commented on issue #6474: [SUPPORT] Hudi Deltastreamer fails to acquire lock with DynamoDB Lock Provider.

2022-08-22 Thread GitBox
nsivabalan commented on issue #6474: URL: https://github.com/apache/hudi/issues/6474#issuecomment-1223375398 are you sure you have attached the right logs. ``` grep "DynamoDBBasedLockProvider" ~/Downloads/logs.txt | wc -l 0 nsb$ grep "TransactionManager"

[jira] [Assigned] (HUDI-4674) change the default value of inputFormat for the MOR table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4674: Assignee: linfey.nie > change the default value of inputFormat for the MOR table >

[jira] [Assigned] (HUDI-4694) Analyze the latest UT/FT runtime

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4694: Assignee: Raymond Xu > Analyze the latest UT/FT runtime > > >

[jira] [Assigned] (HUDI-4125) Add integration tests around bootstrapped Hudi table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-4125: Assignee: (was: Raymond Xu) > Add integration tests around bootstrapped Hudi table >

[jira] [Updated] (HUDI-4674) change the default value of inputFormat for the MOR table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4674: - Reviewers: Raymond Xu > change the default value of inputFormat for the MOR table >

[jira] [Updated] (HUDI-3861) 'path' in CatalogTable#properties failed to be updated when renaming table

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3861: - Reviewers: Raymond Xu > 'path' in CatalogTable#properties failed to be updated when renaming table >

[jira] [Updated] (HUDI-4441) Disbale INFO level logs from tests

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4441: - Sprint: 2022/08/08 (was: 2022/08/22) > Disbale INFO level logs from tests >

[jira] [Updated] (HUDI-2695) [DOCS] Trino Hudi connector on Hudi website

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-2695: - Epic Link: HUDI-2687 > [DOCS] Trino Hudi connector on Hudi website >

[jira] [Updated] (HUDI-4629) Create hive table from existing hoodie Table failed when the table schema is not defined

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4629: - Reviewers: Raymond Xu > Create hive table from existing hoodie Table failed when the table schema is >

[jira] [Updated] (HUDI-4619) The retry mechanism of remotehoodietablefilesystemview needs to be thread safe

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4619: - Reviewers: Raymond Xu > The retry mechanism of remotehoodietablefilesystemview needs to be thread safe >

[jira] [Updated] (HUDI-4615) Fix empty commits being made by deltastreamer with S3EventsSource when there is no data in SQS on starting a new pipeline

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4615: - Reviewers: sivabalan narayanan > Fix empty commits being made by deltastreamer with S3EventsSource when

[jira] [Updated] (HUDI-4431) Fix log file will not roll over to a new file

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4431: - Reviewers: Raymond Xu > Fix log file will not roll over to a new file >

[jira] [Updated] (HUDI-4340) DeltaStreamer bootstrap failed when metrics on caused by DateTimeParseException: Text '00000000000001999' could not be parsed

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4340: - Reviewers: Raymond Xu > DeltaStreamer bootstrap failed when metrics on caused by >

[jira] [Updated] (HUDI-4549) hive sync bundle causes class loader issue

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4549: - Reviewers: Raymond Xu > hive sync bundle causes class loader issue >

[jira] [Updated] (HUDI-4694) Analyze the latest UT/FT runtime

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4694: - Priority: Blocker (was: Major) > Analyze the latest UT/FT runtime > > >

[jira] [Updated] (HUDI-3287) Remove unnecessary deps in hudi-kafka-connect

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3287: - Priority: Major (was: Blocker) > Remove unnecessary deps in hudi-kafka-connect >

[jira] [Updated] (HUDI-4417) Update Hudi Storage docs

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-4417: - Sprint: (was: 2022/08/22) > Update Hudi Storage docs > > >

[jira] [Updated] (HUDI-3287) Remove unnecessary deps in hudi-kafka-connect

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3287: - Component/s: dependencies > Remove unnecessary deps in hudi-kafka-connect >

[jira] [Updated] (HUDI-3287) Remove unnecessary deps in hudi-kafka-connect

2022-08-22 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-3287: - Sprint: Hudi-Sprint-Mar-01 (was: Hudi-Sprint-Mar-01, 2022/08/22) > Remove unnecessary deps in

  1   2   3   4   5   >