[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374727149 ## CI report: * af728e3eeb1fd694eba037bc9a48869831ddb053 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1417

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374725718 ## CI report: * af728e3eeb1fd694eba037bc9a48869831ddb053 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1417

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374724389 ## CI report: * af728e3eeb1fd694eba037bc9a48869831ddb053 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
hudi-bot commented on PR #7621: URL: https://github.com/apache/hudi/pull/7621#issuecomment-1374723047 ## CI report: * 2837b8dc79a5e968f9a15e3f79547dcc7f4b142f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #7608: [HUDI-5503]Optimize flink table factory option check

2023-01-07 Thread GitBox
hudi-bot commented on PR #7608: URL: https://github.com/apache/hudi/pull/7608#issuecomment-1374723003 ## CI report: * 158f8a9c55aecdfe8465e092651edbbd24f911f4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] XuQianJin-Stars commented on pull request #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
XuQianJin-Stars commented on PR #7621: URL: https://github.com/apache/hudi/pull/7621#issuecomment-1374715339 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] XuQianJin-Stars commented on pull request #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
XuQianJin-Stars commented on PR #7621: URL: https://github.com/apache/hudi/pull/7621#issuecomment-1374714684 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7620: [HUDI-5511] Do not clean the CkpMetadata dir when restart the job

2023-01-07 Thread GitBox
SteNicholas commented on code in PR #7620: URL: https://github.com/apache/hudi/pull/7620#discussion_r1064075908 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -92,13 +92,14 @@ public void close() { // --

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374708940 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] davidshtian commented on issue #7591: [SUPPORT] Kinesis Data Analytics Flink1.13 to HUDI

2023-01-07 Thread GitBox
davidshtian commented on issue #7591: URL: https://github.com/apache/hudi/issues/7591#issuecomment-1374699427 > @davidshtian @soumilshah1995 Have you tried 1.13.2 version of the packege _flink-s3-fs-hadoop-1.13.2.jar_? As KDA [supports for Apache Flink version 1.13.2](https://docs.aw

[GitHub] [hudi] hudi-bot commented on pull request #7608: [HUDI-5503]Optimize flink table factory option check

2023-01-07 Thread GitBox
hudi-bot commented on PR #7608: URL: https://github.com/apache/hudi/pull/7608#issuecomment-1374699255 ## CI report: * f7391999a7868e7c97797823cab078a3e42f0bca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] hbgstc123 commented on a diff in pull request #7608: [HUDI-5503]Optimize flink table factory option check

2023-01-07 Thread GitBox
hbgstc123 commented on code in PR #7608: URL: https://github.com/apache/hudi/pull/7608#discussion_r1064076705 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java: ## @@ -88,11 +95,44 @@ public DynamicTableSink createDynamicTableSink(Co

[GitHub] [hudi] hudi-bot commented on pull request #7608: [HUDI-5503]Optimize flink table factory option check

2023-01-07 Thread GitBox
hudi-bot commented on PR #7608: URL: https://github.com/apache/hudi/pull/7608#issuecomment-1374698509 ## CI report: * f7391999a7868e7c97797823cab078a3e42f0bca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] hbgstc123 commented on a diff in pull request #7608: [HUDI-5503]Optimize flink table factory option check

2023-01-07 Thread GitBox
hbgstc123 commented on code in PR #7608: URL: https://github.com/apache/hudi/pull/7608#discussion_r1064076453 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java: ## @@ -69,7 +77,6 @@ public class HoodieTableFactory implements Dynamic

[GitHub] [hudi] hbgstc123 commented on a diff in pull request #7608: [HUDI-5503]Optimize flink table factory option check

2023-01-07 Thread GitBox
hbgstc123 commented on code in PR #7608: URL: https://github.com/apache/hudi/pull/7608#discussion_r1064076345 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/bulk/RowDataKeyGen.java: ## @@ -134,7 +155,9 @@ public HoodieKey getHoodieKey(RowData rowData) {

[GitHub] [hudi] SteNicholas commented on a diff in pull request #7620: [HUDI-5511] Do not clean the CkpMetadata dir when restart the job

2023-01-07 Thread GitBox
SteNicholas commented on code in PR #7620: URL: https://github.com/apache/hudi/pull/7620#discussion_r1064075908 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java: ## @@ -92,13 +92,14 @@ public void close() { // --

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374685640 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-07 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374684926 ## CI report: * a3f8cab6db30b8186e19c3c3ac1c85c0fe3fa63f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] ThinkerLei commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
ThinkerLei commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374684521 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374684204 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] KnightChess commented on a diff in pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-07 Thread GitBox
KnightChess commented on code in PR #7607: URL: https://github.com/apache/hudi/pull/7607#discussion_r1064068273 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -81,10 +80,8 @@ trait ProvidesHoodieConfig extend

[GitHub] [hudi] ThinkerLei commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
ThinkerLei commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374672841 Test failure has nothing to do with this PR, @hudi-bot run azure re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-07 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374649595 ## CI report: * edfcc047ac71663a47813ac4187a523cbd0e5c9e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-07 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374648144 ## CI report: * edfcc047ac71663a47813ac4187a523cbd0e5c9e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-07 Thread GitBox
hudi-bot commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374619773 ## CI report: * c59596637cd44124388717082704db7e7bb8bdaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-07 Thread GitBox
hudi-bot commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374569864 ## CI report: * c59596637cd44124388717082704db7e7bb8bdaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] soumilshah1995 commented on issue #7591: [SUPPORT] Kinesis Data Analytics Flink1.13 to HUDI

2023-01-07 Thread GitBox
soumilshah1995 commented on issue #7591: URL: https://github.com/apache/hudi/issues/7591#issuecomment-1374565548 @davidshtian -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] cxzl25 commented on pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-07 Thread GitBox
cxzl25 commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374546366 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [hudi] xushiyan commented on issue #7487: [SUPPORT] S3 Buckets reached quota limit when reading from hudi tables

2023-01-07 Thread GitBox
xushiyan commented on issue #7487: URL: https://github.com/apache/hudi/issues/7487#issuecomment-1374545829 Is this still happening? pls share more info like what the job is doing when this occurs - is it reading or writing? the logs would tell. It's likely due to a lot of small files. have

[GitHub] [hudi] xushiyan commented on issue #7533: [SUPPORT] Recreate deleted metadata table

2023-01-07 Thread GitBox
xushiyan commented on issue #7533: URL: https://github.com/apache/hudi/issues/7533#issuecomment-1374545025 @szingerpeter @yihua what is the latest state of this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [hudi] xushiyan commented on issue #7494: FileNotFoundException while writing dataframe to local file system

2023-01-07 Thread GitBox
xushiyan commented on issue #7494: URL: https://github.com/apache/hudi/issues/7494#issuecomment-1374544413 > java.io.FileNotFoundException: File file:/tmp/hudi_trips_cow_4 does not exist Likely the file path scheme is not working. pls refer to @jonvex 's complete example above. will

[GitHub] [hudi] xushiyan closed issue #7494: FileNotFoundException while writing dataframe to local file system

2023-01-07 Thread GitBox
xushiyan closed issue #7494: FileNotFoundException while writing dataframe to local file system URL: https://github.com/apache/hudi/issues/7494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [hudi] xushiyan closed issue #7507: [SUPPORT] how to use flink offline with occ

2023-01-07 Thread GitBox
xushiyan closed issue #7507: [SUPPORT] how to use flink offline with occ URL: https://github.com/apache/hudi/issues/7507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [hudi] xushiyan commented on issue #7507: [SUPPORT] how to use flink offline with occ

2023-01-07 Thread GitBox
xushiyan commented on issue #7507: URL: https://github.com/apache/hudi/issues/7507#issuecomment-1374543732 closing this as suggestion was provided. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] xushiyan closed issue #7530: Hudi Log files are increasing in our application day by day

2023-01-07 Thread GitBox
xushiyan closed issue #7530: Hudi Log files are increasing in our application day by day URL: https://github.com/apache/hudi/issues/7530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] xushiyan commented on issue #7530: Hudi Log files are increasing in our application day by day

2023-01-07 Thread GitBox
xushiyan commented on issue #7530: URL: https://github.com/apache/hudi/issues/7530#issuecomment-1374542671 is this the same issue as https://github.com/apache/hudi/issues/7600 ? let's consolidate the discussion in one place. moving the discussion there and will link the issue there and clos

[GitHub] [hudi] hudi-bot commented on pull request #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
hudi-bot commented on PR #7621: URL: https://github.com/apache/hudi/pull/7621#issuecomment-1374542301 ## CI report: * 2837b8dc79a5e968f9a15e3f79547dcc7f4b142f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-07 Thread GitBox
hudi-bot commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374542270 ## CI report: * c59596637cd44124388717082704db7e7bb8bdaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] xushiyan commented on issue #7531: [SUPPORT] table comments not fully supported

2023-01-07 Thread GitBox
xushiyan commented on issue #7531: URL: https://github.com/apache/hudi/issues/7531#issuecomment-1374538276 @jonvex can you look into this please? looks like some config fixes should resolve it -- This is an automated message from the Apache Git Service. To respond to the message, please l

[jira] [Created] (HUDI-5513) Improve documentation for spark-sql write configs

2023-01-07 Thread Jonathan Vexler (Jira)
Jonathan Vexler created HUDI-5513: - Summary: Improve documentation for spark-sql write configs Key: HUDI-5513 URL: https://issues.apache.org/jira/browse/HUDI-5513 Project: Apache Hudi Issue T

[GitHub] [hudi] xushiyan commented on issue #7539: [SUPPORT]IllegalStateException: Trying to access closed classloader

2023-01-07 Thread GitBox
xushiyan commented on issue #7539: URL: https://github.com/apache/hudi/issues/7539#issuecomment-1374525865 @hbgstc123 does this happen every few hours or it only happened once so far? can you try upgrading to 0.12.2 and see how it goes? -- This is an automated message from the Apache Git

[GitHub] [hudi] jomach commented on issue #7565: [SUPPORT] Memory Exception when building BuildProfile

2023-01-07 Thread GitBox
jomach commented on issue #7565: URL: https://github.com/apache/hudi/issues/7565#issuecomment-1374525652 The executors are being killed due to memory exceptions. (OOM) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[jira] [Updated] (HUDI-5485) Improve performance of savepoint with MDT

2023-01-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5485: - Component/s: metadata > Improve performance of savepoint with MDT > --

[jira] [Updated] (HUDI-5485) Improve performance of savepoint with MDT

2023-01-07 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-5485: - Priority: Blocker (was: Critical) > Improve performance of savepoint with MDT > -

[GitHub] [hudi] xushiyan commented on issue #7557: [SUPPORT]: org.apache.hudi.exception.HoodieException: Could not sync using the meta sync class org.apache.hudi.hive.HiveSyncTool

2023-01-07 Thread GitBox
xushiyan commented on issue #7557: URL: https://github.com/apache/hudi/issues/7557#issuecomment-1374523969 > *Hive version : 1.2.1000 Hive 1.x is not supported. pls try upgrade to Hive 2.x or 3.x. Also if you're on hudi 0.11.0, pls consider upgrade to later patch releases like 0.11.1

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374523316 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] xushiyan commented on issue #7565: [SUPPORT] Memory Exception when building BuildProfile

2023-01-07 Thread GitBox
xushiyan commented on issue #7565: URL: https://github.com/apache/hudi/issues/7565#issuecomment-1374522126 ```java inputRecords .mapToPair(record -> Pair.of( new Tuple2<>(record.getPartitionPath(), Option.ofNullable(record.getCurrentLocation())), record))

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374521685 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] xushiyan closed issue #7570: [SUPPORT]Sync hive lost some partitions when submit multiple commits at the same time

2023-01-07 Thread GitBox
xushiyan closed issue #7570: [SUPPORT]Sync hive lost some partitions when submit multiple commits at the same time URL: https://github.com/apache/hudi/issues/7570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [hudi] xushiyan commented on issue #7589: Keep only clustered file(all) after cleaning

2023-01-07 Thread GitBox
xushiyan commented on issue #7589: URL: https://github.com/apache/hudi/issues/7589#issuecomment-1374517461 @maheshguptags what you need is to do savepointing. see https://hudi.apache.org/docs/disaster_recovery For each clustering (replace commit), you just need to trigger a savepoint and

[GitHub] [hudi] xushiyan closed issue #7596: [SUPPORT] java.lang.NoSuchMethodException: org.apache.hudi.utilities.sources.AvroKafkaSource when running HoodieDeltaStreamer

2023-01-07 Thread GitBox
xushiyan closed issue #7596: [SUPPORT] java.lang.NoSuchMethodException: org.apache.hudi.utilities.sources.AvroKafkaSource when running HoodieDeltaStreamer URL: https://github.com/apache/hudi/issues/7596 -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hudi] ThinkerLei commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
ThinkerLei commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374513867 @hudi-bot run azure re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [hudi] xushiyan commented on issue #7600: Hoodie clean is not deleting old files for MOR table

2023-01-07 Thread GitBox
xushiyan commented on issue #7600: URL: https://github.com/apache/hudi/issues/7600#issuecomment-1374513596 @SabyasachiDasTR have you observed any error or warn in logs? it's likely that something is blocking the clean or failing it. Can you search logs and find any statement wrt "clean"? lo

[GitHub] [hudi] xushiyan commented on issue #7602: [SUPPORT] When does the Spark engine's bulk insert mode support bucket index

2023-01-07 Thread GitBox
xushiyan commented on issue #7602: URL: https://github.com/apache/hudi/issues/7602#issuecomment-1374503920 @minihippo can you please advise? it's gonna be a very useful improvement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hudi] xushiyan commented on issue #7617: [SUPPORT] Hudi "write" command doesn't fail when on incompatible partition type, but "read" command fails.

2023-01-07 Thread GitBox
xushiyan commented on issue #7617: URL: https://github.com/apache/hudi/issues/7617#issuecomment-1374501572 @jonvex can you help verify this with 0.12.2 and master version pls? just to confirm if the behavior was fixed. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [hudi] hudi-bot commented on pull request #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
hudi-bot commented on PR #7621: URL: https://github.com/apache/hudi/pull/7621#issuecomment-1374499632 ## CI report: * 2837b8dc79a5e968f9a15e3f79547dcc7f4b142f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374499289 ## CI report: Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated messag

[GitHub] [hudi] hudi-bot commented on pull request #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
hudi-bot commented on PR #7621: URL: https://github.com/apache/hudi/pull/7621#issuecomment-1374498072 ## CI report: * 2837b8dc79a5e968f9a15e3f79547dcc7f4b142f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-07 Thread GitBox
hudi-bot commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374498015 ## CI report: * 1ac267ba9af690ecd47f74f60c34851387aee9eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1408

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374497670 ## CI report: Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated messag

[GitHub] [hudi] hudi-bot commented on pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-07 Thread GitBox
hudi-bot commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374496229 ## CI report: * 1ac267ba9af690ecd47f74f60c34851387aee9eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1408

[GitHub] [hudi] yuzhaojing closed pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
yuzhaojing closed pull request #6732: [HUDI-4148] Add client for hudi table service manager URL: https://github.com/apache/hudi/pull/6732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] yuzhaojing opened a new pull request, #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
yuzhaojing opened a new pull request, #6732: URL: https://github.com/apache/hudi/pull/6732 ### Change Logs Refactor the part of BaseHoodieWriteClient about table service and wrapped it into BaseHoodieTableServiceClient. _About the Public API for the table service part of BaseHo

[jira] [Updated] (HUDI-5512) spark call procedure run_bootstrap missing params cause job fail

2023-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5512: - Labels: pull-request-available (was: ) > spark call procedure run_bootstrap missing params cause

[GitHub] [hudi] KnightChess opened a new pull request, #7621: [HUDI-5512] fix spark call procedure run_bootstrap missing conf and c…

2023-01-07 Thread GitBox
KnightChess opened a new pull request, #7621: URL: https://github.com/apache/hudi/pull/7621 ### Change Logs - According to the init bootstrap table code in `BootstrapExecutor` and `HoodieSparkSqlWriter`, add some conf to hoodie.properties - fix `key_generator_class` can not take ef

[jira] [Created] (HUDI-5512) spark call procedure run_bootstrap missing params cause job fail

2023-01-07 Thread KnightChess (Jira)
KnightChess created HUDI-5512: - Summary: spark call procedure run_bootstrap missing params cause job fail Key: HUDI-5512 URL: https://issues.apache.org/jira/browse/HUDI-5512 Project: Apache Hudi

[GitHub] [hudi] yuzhaojing commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-07 Thread GitBox
yuzhaojing commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374481061 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374470670 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7620: [HUDI-5511] Do not clean the CkpMetadata dir when restart the job

2023-01-07 Thread GitBox
hudi-bot commented on PR #7620: URL: https://github.com/apache/hudi/pull/7620#issuecomment-1374429123 ## CI report: * cd670233392323f8602950a5d2595661b668f3e9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-07 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374428866 ## CI report: * edfcc047ac71663a47813ac4187a523cbd0e5c9e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-07 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374415222 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415