[GitHub] [hudi] Gatsby-Lee commented on issue #4873: Processing time very Slow Updating records into Hudi Dataset(MOR) using AWS Glue

Gatsby-Lee commented on issue #4873: URL: https://github.com/apache/hudi/issues/4873#issuecomment-1053272675 @cafelo-pfdrive I think the compaction might happen based on the compaction strategy. ( default is num_commit ) * compaction.trigger.strategy * metadata.compaction.delt

[GitHub] [hudi] fengjian428 edited a comment on issue #3755: [Delta Streamer] file name mismatch with meta when compaction running

fengjian428 edited a comment on issue #3755: URL: https://github.com/apache/hudi/issues/3755#issuecomment-1053172311 > #4753 could this happens when using cow table? @yihua -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [hudi] fengjian428 commented on issue #3755: [Delta Streamer] file name mismatch with meta when compaction running

fengjian428 commented on issue #3755: URL: https://github.com/apache/hudi/issues/3755#issuecomment-1053172311 > #4753 could this happens when using cow table? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] danny0405 commented on pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

danny0405 commented on pull request #4880: URL: https://github.com/apache/hudi/pull/4880#issuecomment-1053032393 > I skimmed through the patch. Looks like, we are looking to fix only a specific scenario. > > insert record with preCombine value 100. delete record with preCombine valu

[GitHub] [hudi] danny0405 commented on a change in pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

danny0405 commented on a change in pull request #4880: URL: https://github.com/apache/hudi/pull/4880#discussion_r815381860 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java ## @@ -153,9 +154,29 @@ protected void processN

[GitHub] [hudi] cafelo-pfdrive commented on issue #4873: Processing time very Slow Updating records into Hudi Dataset(MOR) using AWS Glue

cafelo-pfdrive commented on issue #4873: URL: https://github.com/apache/hudi/issues/4873#issuecomment-1052630279 @nsivabalan, here it goes the .hoodie folder Please let me know if this is the information that you are asking for ![image](https://user-images.githubusercontent.com/931568

[GitHub] [hudi] cafelo-pfdrive commented on issue #4873: Processing time very Slow Updating records into Hudi Dataset(MOR) using AWS Glue

cafelo-pfdrive commented on issue #4873: URL: https://github.com/apache/hudi/issues/4873#issuecomment-1052626249 @Gatsby-Lee, @nsivabalan. Thanks much for the explanation, base in your recommendation I changed the Index to 'hoodie.index.type': 'SIMPLE' and add a new column in the partition

[GitHub] [hudi] hudi-bot removed a comment on pull request #4907: [WIP][CI Test Only][HUDI-1180] Upgrade HBase to 2.4.9

hudi-bot removed a comment on pull request #4907: URL: https://github.com/apache/hudi/pull/4907#issuecomment-1052512307 ## CI report: * 55db32bf4b6aa3796be90879815328ae376dd606 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4907: [WIP][CI Test Only][HUDI-1180] Upgrade HBase to 2.4.9

hudi-bot commented on pull request #4907: URL: https://github.com/apache/hudi/pull/4907#issuecomment-1052617166 ## CI report: * 0735d18736639df43d4472580e64234561ea6ff4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] yihua commented on pull request #4494: [WIP][HUDI-3127] Adjust table config for HFile bootstrap index for Trino

yihua commented on pull request #4494: URL: https://github.com/apache/hudi/pull/4494#issuecomment-1052513866 @nsivabalan @codope we might not need this after HBase upgrade to 2.x. Please do not review this PR for now. -- This is an automated message from the Apache Git Service. To respo

[GitHub] [hudi] hudi-bot removed a comment on pull request #4907: [WIP][CI Test Only][HUDI-1180] Upgrade HBase to 2.4.9

hudi-bot removed a comment on pull request #4907: URL: https://github.com/apache/hudi/pull/4907#issuecomment-1052508690 ## CI report: * 55db32bf4b6aa3796be90879815328ae376dd606 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4907: [WIP][CI Test Only][HUDI-1180] Upgrade HBase to 2.4.9

hudi-bot commented on pull request #4907: URL: https://github.com/apache/hudi/pull/4907#issuecomment-1052512307 ## CI report: * 55db32bf4b6aa3796be90879815328ae376dd606 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4907: [WIP][CI Test Only][HUDI-1180] Upgrade HBase to 2.4.9

hudi-bot removed a comment on pull request #4907: URL: https://github.com/apache/hudi/pull/4907#issuecomment-1051421561 ## CI report: * 55db32bf4b6aa3796be90879815328ae376dd606 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4907: [WIP][CI Test Only][HUDI-1180] Upgrade HBase to 2.4.9

hudi-bot commented on pull request #4907: URL: https://github.com/apache/hudi/pull/4907#issuecomment-1052508690 ## CI report: * 55db32bf4b6aa3796be90879815328ae376dd606 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] yihua commented on issue #3755: [Delta Streamer] file name mismatch with meta when compaction running

yihua commented on issue #3755: URL: https://github.com/apache/hudi/issues/3755#issuecomment-1052473596 @fengjian428 Sorry for the late reply. Recently we've triaged an issue related to wrong commit metadata that affects metadata table, caused by RDD caching and the old logic of deriving

[GitHub] [hudi] yihua commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

yihua commented on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-1052459014 > @xushiyan @yihua : with sql-dml, how are timeline server instantiated? Is it the same flow as sparkdatasource. @yihua : Do we know if the timeline server is running in this case ? or i

[GitHub] [hudi] hudi-bot removed a comment on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot removed a comment on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1052246021 ## CI report: * c6f524ea6bf6800a797f9173c4034ce8d111a7d5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1052318257 ## CI report: * d3b3e052e3565fe44174dfae73938bfaa44faee4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot removed a comment on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1052242175 ## CI report: * c6f524ea6bf6800a797f9173c4034ce8d111a7d5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1052246021 ## CI report: * c6f524ea6bf6800a797f9173c4034ce8d111a7d5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1052242175 ## CI report: * c6f524ea6bf6800a797f9173c4034ce8d111a7d5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4724: [HUDI-2815] add partial overwrite payload to support partial overwrit…

hudi-bot removed a comment on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1051022414 ## CI report: * c6f524ea6bf6800a797f9173c4034ce8d111a7d5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052224327 ## CI report: * 2243884a4617e11f0c2d675bc973c644744c3a1e UNKNOWN * 58a02eb9851c09f765371bd6eb71727def188baf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052154487 ## CI report: * d6d908a21aab91c8d05f65f3d61e357eaf0badde Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052203919 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * 78e86dd1953cc4d6bf10ca808a7bcffe22b4b587 UNKNOWN * 4ff0d57275e8f907d945c60bd93c2bef227c7c3d Azur

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052133893 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

hudi-bot commented on pull request #4840: URL: https://github.com/apache/hudi/pull/4840#issuecomment-1052175499 ## CI report: * 260c29fa4d9e1d0e45e9ee771d51ed3f6b2f7c5f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

hudi-bot removed a comment on pull request #4840: URL: https://github.com/apache/hudi/pull/4840#issuecomment-1052119893 ## CI report: * 43859892f433775b00097e573b3c4e9971f8f3fb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052154487 ## CI report: * d6d908a21aab91c8d05f65f3d61e357eaf0badde Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052134600 ## CI report: * d6d908a21aab91c8d05f65f3d61e357eaf0badde Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4812: [HUDI-2761] Fixing timeline server for repeated refreshes

hudi-bot commented on pull request #4812: URL: https://github.com/apache/hudi/pull/4812#issuecomment-1052138418 ## CI report: * d67e7d11d3eba65df4a40636629c2a73796ab5d6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4812: [HUDI-2761] Fixing timeline server for repeated refreshes

hudi-bot removed a comment on pull request #4812: URL: https://github.com/apache/hudi/pull/4812#issuecomment-1052118652 ## CI report: * d59ff5d0d37425f039f77c7c4bb02b41ba0a7e38 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] matthiasdg commented on issue #3868: [SUPPORT] hive syncing with `--spark-datasource` (first title was: Querying hudi datasets from standalone metastore)

matthiasdg commented on issue #3868: URL: https://github.com/apache/hudi/issues/3868#issuecomment-1052138018 I'll try to take a look somewhere in the coming week! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hudi] xiarixiaoyao edited a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

xiarixiaoyao edited a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1050837239 @bvaradar @codope @vinothchandar @xushiyan could you pls help me review this pr, this pr stacked on hudi-2569, all sql support can be find in TestSpark3DDL.scala

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052134600 ## CI report: * d6d908a21aab91c8d05f65f3d61e357eaf0badde Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052133631 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052133893 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052132941 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052130264 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052133631 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052130532 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052132941 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052130532 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052016155 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hud

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052130264 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052129201 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot commented on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1052129201 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4441: [HUDI-3085] improve bulk insert partitioner abstraction

hudi-bot removed a comment on pull request #4441: URL: https://github.com/apache/hudi/pull/4441#issuecomment-1050443998 ## CI report: * fb474d121dc178d2284c2d8a80b7f9380aa902a8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan edited a comment on pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

nsivabalan edited a comment on pull request #4880: URL: https://github.com/apache/hudi/pull/4880#issuecomment-1052124364 @danny0405 : Can you update the desc of the PR. @danny0405 @xushiyan : I looked at the jira linked and w/o getting into implementation details (I have not looked i

[GitHub] [hudi] hudi-bot commented on pull request #4468: [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas

hudi-bot commented on pull request #4468: URL: https://github.com/apache/hudi/pull/4468#issuecomment-1052128438 ## CI report: * c032433e87c4eccf92b90cea68e2811ef2a0f225 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4468: [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas

hudi-bot removed a comment on pull request #4468: URL: https://github.com/apache/hudi/pull/4468#issuecomment-1052115832 ## CI report: * 2461ceb7bb042694782128370df1e96f05a71391 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

nsivabalan commented on pull request #4880: URL: https://github.com/apache/hudi/pull/4880#issuecomment-1052128140 I skimmed through the patch. Looks like, we are looking to fix only a specific scenario. insert record with preCombine value 100. delete record with preCombine val

[GitHub] [hudi] nsivabalan commented on a change in pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

nsivabalan commented on a change in pull request #4880: URL: https://github.com/apache/hudi/pull/4880#discussion_r815311951 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java ## @@ -153,9 +154,29 @@ protected void process

[GitHub] [hudi] nsivabalan commented on pull request #4880: [HUDI-2752] The MOR DELETE block breaks the event time sequence of CDC

nsivabalan commented on pull request #4880: URL: https://github.com/apache/hudi/pull/4880#issuecomment-1052124364 @danny0405 : Can you update the desc of the PR. @danny0405 @xushiyan : I looked at the jira linked and w/o getting into implementation details (I have not looked into the

[GitHub] [hudi] nsivabalan commented on pull request #4902: [HUDI-3497] Adding Datatable validator tool

nsivabalan commented on pull request #4902: URL: https://github.com/apache/hudi/pull/4902#issuecomment-1052121243 @zhangyue19921010 : Can you help review this as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] nsivabalan commented on pull request #4878: [HUDI-3465] Add validation of column stats and bloom filters in HoodieMetadataTableValidator

nsivabalan commented on pull request #4878: URL: https://github.com/apache/hudi/pull/4878#issuecomment-1052120814 I have quite a few PRs and issues to catch up. I will let @zhangyue19921010 review this. Once he stamps, I can help land it. -- This is an automated message from the Apa

[GitHub] [hudi] nsivabalan commented on issue #4896: [SUPPORT] Metadata Table causes missing data.

nsivabalan commented on issue #4896: URL: https://github.com/apache/hudi/issues/4896#issuecomment-1052120522 ok. when metadata table is enabled, and metadata.clean.async is false: can you post the contents of .hoodie and .hoodie/metadata.hoodie and we can go from there. Ens

[GitHub] [hudi] nsivabalan commented on issue #4899: clustering cause Duplicate Records in Merge on Read [SUPPORT]

nsivabalan commented on issue #4899: URL: https://github.com/apache/hudi/issues/4899#issuecomment-1052120084 got it, thanks! please re-open if you run into any issues. thanks for bringing up the issue. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [hudi] nsivabalan closed issue #4899: clustering cause Duplicate Records in Merge on Read [SUPPORT]

nsivabalan closed issue #4899: URL: https://github.com/apache/hudi/issues/4899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...

[GitHub] [hudi] hudi-bot commented on pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

hudi-bot commented on pull request #4840: URL: https://github.com/apache/hudi/pull/4840#issuecomment-1052119893 ## CI report: * 43859892f433775b00097e573b3c4e9971f8f3fb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

hudi-bot removed a comment on pull request #4840: URL: https://github.com/apache/hudi/pull/4840#issuecomment-1052119233 ## CI report: * 43859892f433775b00097e573b3c4e9971f8f3fb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

hudi-bot commented on pull request #4840: URL: https://github.com/apache/hudi/pull/4840#issuecomment-1052119233 ## CI report: * 43859892f433775b00097e573b3c4e9971f8f3fb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

hudi-bot removed a comment on pull request #4840: URL: https://github.com/apache/hudi/pull/4840#issuecomment-1048875213 ## CI report: * 43859892f433775b00097e573b3c4e9971f8f3fb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[hudi] branch master updated (1379300 -> c77b259)

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 1379300 [HUDI-3483] Adding insert override nodes to integ test suite and few clean ups (#4895) add c77b259 [

[GitHub] [hudi] nsivabalan merged pull request #4860: [HUDI-2439] Remove SparkBoundedInMemoryExecutor

nsivabalan merged pull request #4860: URL: https://github.com/apache/hudi/pull/4860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] nsivabalan commented on a change in pull request #4860: [HUDI-2439] Remove SparkBoundedInMemoryExecutor

nsivabalan commented on a change in pull request #4860: URL: https://github.com/apache/hudi/pull/4860#discussion_r815308861 ## File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeUnmergedRecordReader.java ## @@ -74,7 +78,7 @@ public RealtimeUnmerged

[GitHub] [hudi] nsivabalan commented on pull request #4807: [HUDI-3387] Make async timeline server by default

nsivabalan commented on pull request #4807: URL: https://github.com/apache/hudi/pull/4807#issuecomment-1052118978 @xushiyan : were you able to make any progress on this end. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[hudi] branch master updated: [HUDI-3483] Adding insert override nodes to integ test suite and few clean ups (#4895)

This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 1379300 [HUDI-3483] Adding insert override nod

[GitHub] [hudi] nsivabalan commented on pull request #4895: [HUDI-3483] Adding insert override nodes to integ test suite and few clean ups

nsivabalan commented on pull request #4895: URL: https://github.com/apache/hudi/pull/4895#issuecomment-1052118700 Ran tests locally and verified. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] nsivabalan merged pull request #4895: [HUDI-3483] Adding insert override nodes to integ test suite and few clean ups

nsivabalan merged pull request #4895: URL: https://github.com/apache/hudi/pull/4895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [hudi] hudi-bot commented on pull request #4812: [HUDI-2761] Fixing timeline server for repeated refreshes

hudi-bot commented on pull request #4812: URL: https://github.com/apache/hudi/pull/4812#issuecomment-1052118652 ## CI report: * d59ff5d0d37425f039f77c7c4bb02b41ba0a7e38 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4812: [HUDI-2761] Fixing timeline server for repeated refreshes

hudi-bot removed a comment on pull request #4812: URL: https://github.com/apache/hudi/pull/4812#issuecomment-1052118178 ## CI report: * d59ff5d0d37425f039f77c7c4bb02b41ba0a7e38 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] nsivabalan commented on a change in pull request #4840: [HUDI-2917] rollback insert data appended to log file when using Hbase Index

nsivabalan commented on a change in pull request #4840: URL: https://github.com/apache/hudi/pull/4840#discussion_r815308544 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java ## @@ -182,14 +182,30 @@ public abst

[GitHub] [hudi] hudi-bot commented on pull request #4812: [HUDI-2761] Fixing timeline server for repeated refreshes

hudi-bot commented on pull request #4812: URL: https://github.com/apache/hudi/pull/4812#issuecomment-1052118178 ## CI report: * d59ff5d0d37425f039f77c7c4bb02b41ba0a7e38 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4812: [HUDI-2761] Fixing timeline server for repeated refreshes

hudi-bot removed a comment on pull request #4812: URL: https://github.com/apache/hudi/pull/4812#issuecomment-1051027082 ## CI report: * d59ff5d0d37425f039f77c7c4bb02b41ba0a7e38 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4468: [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas

hudi-bot commented on pull request #4468: URL: https://github.com/apache/hudi/pull/4468#issuecomment-1052115832 ## CI report: * 2461ceb7bb042694782128370df1e96f05a71391 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4468: [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas

hudi-bot removed a comment on pull request #4468: URL: https://github.com/apache/hudi/pull/4468#issuecomment-1052115368 ## CI report: * 2461ceb7bb042694782128370df1e96f05a71391 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4468: [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas

hudi-bot removed a comment on pull request #4468: URL: https://github.com/apache/hudi/pull/4468#issuecomment-1051316141 ## CI report: * 2461ceb7bb042694782128370df1e96f05a71391 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4468: [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas

hudi-bot commented on pull request #4468: URL: https://github.com/apache/hudi/pull/4468#issuecomment-1052115368 ## CI report: * 2461ceb7bb042694782128370df1e96f05a71391 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot commented on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1052097829 ## CI report: * 834d069d45d0b3bb843c9266398b30834a65d7f0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot removed a comment on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1052005591 ## CI report: * f0dbae2d1d08c81aee0a6890d789fe0784a11065 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1052016155 ## CI report: * 52be34d7d5e025180415c46e64a3e2145c29e498 UNKNOWN * e6fb8cba2ee5c9129d6ed88ff3cf137e600cb14d Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051928469 ## CI report: * 2685177494a54601e91a21d849d2ffd6e284f4ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot removed a comment on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot removed a comment on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1052001884 ## CI report: * f0dbae2d1d08c81aee0a6890d789fe0784a11065 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot commented on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1052005591 ## CI report: * f0dbae2d1d08c81aee0a6890d789fe0784a11065 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot commented on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot commented on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1052001884 ## CI report: * f0dbae2d1d08c81aee0a6890d789fe0784a11065 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot removed a comment on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1051896118 ## CI report: * f0dbae2d1d08c81aee0a6890d789fe0784a11065 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051878614 ## CI report: * 2685177494a54601e91a21d849d2ffd6e284f4ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051928469 ## CI report: * 2685177494a54601e91a21d849d2ffd6e284f4ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot commented on pull request #4913: [WIP][HUDI-1517] create marker file for every log file

hudi-bot commented on pull request #4913: URL: https://github.com/apache/hudi/pull/4913#issuecomment-1051907113 ## CI report: * 6816a4b47b88108172b46fece160e4e078345687 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] hudi-bot removed a comment on pull request #4913: [WIP][HUDI-1517] create marker file for every log file

hudi-bot removed a comment on pull request #4913: URL: https://github.com/apache/hudi/pull/4913#issuecomment-1051787441 ## CI report: * 875ec8b00cd379e669498fe7575503b192f0de5e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot removed a comment on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot removed a comment on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1051811749 ## CI report: * f275704b3dc2fbe99be692bc8c4d2cef383664f0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/r

[GitHub] [hudi] hudi-bot commented on pull request #4911: [HUDI-3460]Flink TM memory Optimization

hudi-bot commented on pull request #4911: URL: https://github.com/apache/hudi/pull/4911#issuecomment-1051896118 ## CI report: * f0dbae2d1d08c81aee0a6890d789fe0784a11065 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?b

[GitHub] [hudi] Gatsby-Lee edited a comment on issue #4311: Duplicate Records in Merge on Read [SUPPORT]

Gatsby-Lee edited a comment on issue #4311: URL: https://github.com/apache/hudi/issues/4311#issuecomment-1051883576 @scxwhite hi, have you tried with Hudi Metadata Table disabled? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] Gatsby-Lee commented on issue #4311: Duplicate Records in Merge on Read [SUPPORT]

Gatsby-Lee commented on issue #4311: URL: https://github.com/apache/hudi/issues/4311#issuecomment-1051883576 @scxwhite hi, have you tried with Hudi Metadata Table disabled? In my case, I had the same issue when I enabled "Hudi Metadata Table" After I disabled it, the issue was reso

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051878614 ## CI report: * 2685177494a54601e91a21d849d2ffd6e284f4ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051875111 ## CI report: * 2685177494a54601e91a21d849d2ffd6e284f4ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

[GitHub] [hudi] Gatsby-Lee commented on issue #4713: [SUPPORT] Executor OOM upserting 20M records from Kafka

Gatsby-Lee commented on issue #4713: URL: https://github.com/apache/hudi/issues/4713#issuecomment-1051878036 @stym06 try to set "maxOffsetsPerTrigger" when you init Spark to prevent OOM by controlling the max fetched msg. -- This is an automated message from the Apache Git Servic

[GitHub] [hudi] hudi-bot commented on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot commented on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051875111 ## CI report: * 2685177494a54601e91a21d849d2ffd6e284f4ba Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4910: [RFC-33] [HUDI-2429][Stacked on HUDI-2560] Support full Schema evolution for Spark

hudi-bot removed a comment on pull request #4910: URL: https://github.com/apache/hudi/pull/4910#issuecomment-1051871486 ## CI report: * 758d417cc8f02537d8174f19c904c062b0873646 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/

  1   2   >