[GitHub] [hudi] majian1998 opened a new pull request, #9652: [MINOR] Ensure other table services with correct path are not affect…
majian1998 opened a new pull request, #9652: URL: https://github.com/apache/hudi/pull/9652 …ed by table with wrong path. ### Change Logs Filtering tables with incorrect paths in the multi table service. ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update The current multi table service supports executing table service for multiple tables by specifying them through hoodie.tableservice.tablesToServe. When an incorrect path is provided for a table or a table is deleted but the asynchronous table service is not yet aware of it, the current implementation causes the entire Spark job to fail, impacting the execution of other tables. It is expected that this should not impact the table service of other tables with correctly configured paths. In order to address this issue, I will introduce a check to skip such cases and log the details of tables that do not conform to Hudi's requirements using Log.error. A question for discussion: What kind of exceptions do you think should impact the table service of other tables in the same batch, and what kind should not? ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HUDI-6833) Add field for tracking log files from failed commit in rollback metadata
Lokesh Jain created HUDI-6833: - Summary: Add field for tracking log files from failed commit in rollback metadata Key: HUDI-6833 URL: https://issues.apache.org/jira/browse/HUDI-6833 Project: Apache Hudi Issue Type: Bug Reporter: Lokesh Jain Assignee: Lokesh Jain The Jira aims to add field for tracking log files from failed commit in the rollback metadata. The fix for using this field would be tracked in HUDI-6758. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HUDI-6832) When one of table paths is incorrect, ensure that other table services are not affected.
Ma Jian created HUDI-6832:
-
Summary: When one of table paths is incorrect, ensure that other
table services are not affected.
Key: HUDI-6832
URL: https://issues.apache.org/jira/browse/HUDI-6832
Project: Apache Hudi
Issue Type: Bug
Reporter: Ma Jian
Attachments: image-2023-09-08-14-49-44-564.png
The current multi table service supports executing table service for multiple
tables by specifying them through {{{}hoodie.tableservice.tablesToServe{}}}.
When an incorrect path is provided for a table or a table is deleted but the
asynchronous table service is not yet aware of it, the current implementation
causes the entire Spark job to fail, impacting the execution of other tables.
It is expected that this should not impact the table service of other tables
with correctly configured paths.
!image-2023-09-08-14-49-44-564.png!
In order to address this issue, I will introduce a check to skip such cases and
log the details of tables that do not conform to Hudi's requirements using
Log.error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HUDI-6832) When one of table paths is incorrect, ensure that other table services are not affected.
[
https://issues.apache.org/jira/browse/HUDI-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ma Jian updated HUDI-6832:
--
Description:
The current multi table service supports executing table service for multiple
tables by specifying them through {{{}hoodie.tableservice.tablesToServe{}}}.
When an incorrect path is provided for a table or a table is deleted but the
asynchronous table service is not yet aware of it, the current implementation
causes the entire Spark job to fail, impacting the execution of other tables.
It is expected that this should not impact the table service of other tables
with correctly configured paths.
!image-2023-09-08-14-49-44-564.png|width=840,height=111!
In order to address this issue, I will introduce a check to skip such cases and
log the details of tables that do not conform to Hudi's requirements using
Log.error.
was:
The current multi table service supports executing table service for multiple
tables by specifying them through {{{}hoodie.tableservice.tablesToServe{}}}.
When an incorrect path is provided for a table or a table is deleted but the
asynchronous table service is not yet aware of it, the current implementation
causes the entire Spark job to fail, impacting the execution of other tables.
It is expected that this should not impact the table service of other tables
with correctly configured paths.
!image-2023-09-08-14-49-44-564.png!
In order to address this issue, I will introduce a check to skip such cases and
log the details of tables that do not conform to Hudi's requirements using
Log.error.
> When one of table paths is incorrect, ensure that other table services are
> not affected.
>
>
> Key: HUDI-6832
> URL: https://issues.apache.org/jira/browse/HUDI-6832
> Project: Apache Hudi
> Issue Type: Bug
>Reporter: Ma Jian
>Priority: Minor
> Attachments: image-2023-09-08-14-49-44-564.png
>
>
> The current multi table service supports executing table service for multiple
> tables by specifying them through {{{}hoodie.tableservice.tablesToServe{}}}.
> When an incorrect path is provided for a table or a table is deleted but the
> asynchronous table service is not yet aware of it, the current implementation
> causes the entire Spark job to fail, impacting the execution of other tables.
> It is expected that this should not impact the table service of other tables
> with correctly configured paths.
> !image-2023-09-08-14-49-44-564.png|width=840,height=111!
> In order to address this issue, I will introduce a check to skip such cases
> and log the details of tables that do not conform to Hudi's requirements
> using Log.error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hudi] stream2000 commented on a diff in pull request #9651: [HUDI-6336] Support flink timeline-based ckp metadata
stream2000 commented on code in PR #9651:
URL: https://github.com/apache/hudi/pull/9651#discussion_r1319439232
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##
@@ -418,6 +418,12 @@ public class HoodieWriteConfig extends HoodieConfig {
.sinceVersion("0.9.0")
.withDocumentation(MarkerType.class);
+ public static final ConfigProperty
TIMELINE_SERVER_BASED_FLINK_CKP_METADATA = ConfigProperty
Review Comment:
make it defaults to true to run IT and UT tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] stream2000 opened a new pull request, #9651: [HUDI-6336] Support flink timeline-based ckp metadata
stream2000 opened a new pull request, #9651: URL: https://github.com/apache/hudi/pull/9651 ### Change Logs Support flink timeline-based ckp metadata to reduce file system listing ### Impact taskmanagers will read ckp metadata from timeline server ### Risk level (write none, low medium or high below) low ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9649: [HUDI-6826] Port BloomFilter related classes from Hadoop library
hudi-bot commented on PR #9649: URL: https://github.com/apache/hudi/pull/9649#issuecomment-1711152267 ## CI report: * 3d62171eaeb3cc64a5878462d9bc0606b571770b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19725) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (HUDI-6725) Support efficient completion time queries on the timeline
[ https://issues.apache.org/jira/browse/HUDI-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6725. Resolution: Fixed > Support efficient completion time queries on the timeline > - > > Key: HUDI-6725 > URL: https://issues.apache.org/jira/browse/HUDI-6725 > Project: Apache Hudi > Issue Type: Improvement > Components: core >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > > The basic idea is we do a eager loading of the completion time on archived > timeline, for example, the last 3 days, and all the completed instants of the > active timeline. > If a query is asking about a completion time earlier than that time range, > just do a lazy look up on the archvied timeline. > > Probably we would write a completion time loader. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] hudi-bot commented on pull request #9645: [HUDI-6820] Writer client close(3)
hudi-bot commented on PR #9645: URL: https://github.com/apache/hudi/pull/9645#issuecomment-1711098936 ## CI report: * 1aa4f82d4a4640371d8e62ed76599897d1a2c8dc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19721) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19720) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9644: [HUDI-6820] Closing Write clients (2)
hudi-bot commented on PR #9644: URL: https://github.com/apache/hudi/pull/9644#issuecomment-1711091640 ## CI report: * ef1e29705531b56996e35b1b9137ac9318dc1677 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19719) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9648: [HUDI-6820] Triage ci timeouts(3)
hudi-bot commented on PR #9648: URL: https://github.com/apache/hudi/pull/9648#issuecomment-1711062223 ## CI report: * e2574f456376f0be5a7ed774f5468f795b3bdcf4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19724) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9553: [HUDI-1517][HUDI-6758][HUDI-6761] Adding support for per-logfile marker to track all log files added by a commit and to assist with
danny0405 commented on code in PR #9553:
URL: https://github.com/apache/hudi/pull/9553#discussion_r1319337535
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/utils/CommitMetadataUtils.java:
##
@@ -0,0 +1,225 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hudi.client.utils;
+
+import org.apache.hudi.common.config.SerializableConfiguration;
+import org.apache.hudi.common.data.HoodiePairData;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.common.fs.FSUtils;
+import org.apache.hudi.common.function.SerializableBiFunction;
+import org.apache.hudi.common.function.SerializableFunction;
+import org.apache.hudi.common.function.SerializablePairFunction;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.model.HoodieDeltaWriteStat;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieTableType;
+import org.apache.hudi.common.model.HoodieWriteStat;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.common.util.collection.Pair;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.table.HoodieTable;
+import org.apache.hudi.table.marker.WriteMarkers;
+import org.apache.hudi.table.marker.WriteMarkersFactory;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+public class CommitMetadataUtils {
+
+ /* In spark mor table, task retries may generate log files which are not
included in write status.
+ * We need to add these to CommitMetadata so that it will be synced to MDT.
+ */
+ public static HoodieCommitMetadata
reconcileMetadataForMissingFiles(HoodieTable table, String commitActionType,
String instantTime,
+
HoodieCommitMetadata commitMetadata, HoodieWriteConfig config,
+
HoodieEngineContext context, Configuration hadoopConf, String
classNameForContext) throws IOException {
+if
(!table.getMetaClient().getTableConfig().getTableType().equals(HoodieTableType.MERGE_ON_READ)
+|| !commitActionType.equals(HoodieActiveTimeline.DELTA_COMMIT_ACTION))
{
+ return commitMetadata;
+}
+
+WriteMarkers markers = WriteMarkersFactory.get(config.getMarkersType(),
table, instantTime);
+// if there is log files in this delta commit, we search any invalid log
files generated by failed spark task
+boolean hasLogFileInDeltaCommit = commitMetadata.getPartitionToWriteStats()
+.values().stream().flatMap(List::stream)
+.anyMatch(writeStat -> FSUtils.isLogFile(new
Path(config.getBasePath(), writeStat.getPath()).getName()));
+if (hasLogFileInDeltaCommit) { // skip for COW table
+ // get all log files generated by makers
+ Set allLogFilesMarkerPath = new
HashSet<>(markers.getAppendedLogPaths(context,
config.getFinalizeWriteParallelism()));
+ Set logFilesMarkerPath = new HashSet<>();
+ allLogFilesMarkerPath.stream().filter(logFilePath ->
!logFilePath.endsWith("cdc")).forEach(logFilePath ->
logFilesMarkerPath.add(logFilePath));
+
+ // remove valid log files
+ for (Map.Entry> partitionAndWriteStats :
commitMetadata.getPartitionToWriteStats().entrySet()) {
+for (HoodieWriteStat hoodieWriteStat :
partitionAndWriteStats.getValue()) {
+ logFilesMarkerPath.remove(hoodieWriteStat.getPath());
+}
+ }
+
+ // remaining are log files generated by retried spark task, let's
generate write stat for them
+ if (logFilesMarkerPath.size() > 0) {
+SerializableConfiguration serializableConfiguration = new
SerializableConfiguration
[GitHub] [hudi] hudi-bot commented on pull request #9646: [HUDI-6820] Running CI to triage time outs (1)
hudi-bot commented on PR #9646: URL: https://github.com/apache/hudi/pull/9646#issuecomment-1711052406 ## CI report: * 847f94e17ff7a93d133056f75db59daf54fd45a4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19722) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9643: [HUDI-6820] Write client closure in tests (1)
hudi-bot commented on PR #9643: URL: https://github.com/apache/hudi/pull/9643#issuecomment-1711052360 ## CI report: * bb8f4bce70af2ec873dbfefea43a4a04c55307cf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19718) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19717) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] Zouxxyy closed pull request #9554: [HUDI-6760] Add SelfDescribingInputFormatInterface for hive FileInput…
Zouxxyy closed pull request #9554: [HUDI-6760] Add SelfDescribingInputFormatInterface for hive FileInput… URL: https://github.com/apache/hudi/pull/9554 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
danny0405 commented on code in PR #9650: URL: https://github.com/apache/hudi/pull/9650#discussion_r1319320294 ## hudi-gcp/src/main/java/org/apache/hudi/gcp/bigquery/HoodieBigQuerySyncClient.java: ## @@ -94,8 +94,9 @@ public void createTableUsingBqManifestFile(String tableName, String bqManifestFi } String query = String.format( - "CREATE EXTERNAL TABLE `%s.%s` %s OPTIONS (%s " + "CREATE EXTERNAL TABLE `%s.%s.%s` %s OPTIONS (%s " + "uris=[\"%s\"], format=\"PARQUET\", file_set_spec_type=\"NEW_LINE_DELIMITED_MANIFEST\")", + projectId, Review Comment: Got it, can we write some tests for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9642: [HUDI-6820] Close write clients in tests
hudi-bot commented on PR #9642: URL: https://github.com/apache/hudi/pull/9642#issuecomment-1711026303 ## CI report: * c4e998719c4af415efad6d5aeb84e5705c3761c0 UNKNOWN * e4cf00bca6148ce175d7d6a2d57d9d129258be18 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19728) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19727) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9643: [HUDI-6820] Write client closure in tests (1)
hudi-bot commented on PR #9643: URL: https://github.com/apache/hudi/pull/9643#issuecomment-1711021846 ## CI report: * bb8f4bce70af2ec873dbfefea43a4a04c55307cf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19718) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19717) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9642: [HUDI-6820] Close write clients in tests
hudi-bot commented on PR #9642: URL: https://github.com/apache/hudi/pull/9642#issuecomment-1711021826 ## CI report: * c4e998719c4af415efad6d5aeb84e5705c3761c0 UNKNOWN * bb8f4bce70af2ec873dbfefea43a4a04c55307cf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19717) * e4cf00bca6148ce175d7d6a2d57d9d129258be18 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19727) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19728) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #9642: [HUDI-6820] Close write clients in tests
nsivabalan commented on PR #9642: URL: https://github.com/apache/hudi/pull/9642#issuecomment-1711008722 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] jiaojietao commented on issue #9636: How to use Kerberos in Hudi integrated Flink?data cannot be written to hdfs.
jiaojietao commented on issue #9636: URL: https://github.com/apache/hudi/issues/9636#issuecomment-1711004123 Can you write the group:user to which the path "tmp/hudi_flink/t1" belongs? Can you take a screenshot and see if your ticket should be configured as' hive:hive ' -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9642: [HUDI-6820] Close write clients in tests
hudi-bot commented on PR #9642: URL: https://github.com/apache/hudi/pull/9642#issuecomment-1710995082 ## CI report: * c4e998719c4af415efad6d5aeb84e5705c3761c0 UNKNOWN * fe6adfb876c41ab5ed8c261e661f9dfdf147826a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19704) * bb8f4bce70af2ec873dbfefea43a4a04c55307cf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19717) * e4cf00bca6148ce175d7d6a2d57d9d129258be18 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] jp0317 commented on a diff in pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
jp0317 commented on code in PR #9650: URL: https://github.com/apache/hudi/pull/9650#discussion_r1319282103 ## hudi-gcp/src/main/java/org/apache/hudi/gcp/bigquery/HoodieBigQuerySyncClient.java: ## @@ -94,8 +94,9 @@ public void createTableUsingBqManifestFile(String tableName, String bqManifestFi } String query = String.format( - "CREATE EXTERNAL TABLE `%s.%s` %s OPTIONS (%s " + "CREATE EXTERNAL TABLE `%s.%s.%s` %s OPTIONS (%s " + "uris=[\"%s\"], format=\"PARQUET\", file_set_spec_type=\"NEW_LINE_DELIMITED_MANIFEST\")", + projectId, Review Comment: Thanks for your review! Here the project id can be skipped if the sync and data are in the same project, otherwise it's required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[ https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762925#comment-17762925 ] Lin Liu commented on HUDI-5807: --- Confirmed that we can set the property either by modifying the code, or use external config file hudi-defaults.conf. After that, the partition-path still shows well in the query. Next step: Confirm with the team if this is the expected behavior. If yes, then this issue has been fixed somehow; otherwise, we reproduced the issue and can start to debug. Note that the problem description in the task is not clear enough. We should add sufficient info for task owner to test/reproduce. > HoodieSparkParquetReader is not appending partition-path values > --- > > Key: HUDI-5807 > URL: https://issues.apache.org/jira/browse/HUDI-5807 > Project: Apache Hudi > Issue Type: Bug > Components: spark >Affects Versions: 0.13.0 >Reporter: Alexey Kudinkin >Assignee: Lin Liu >Priority: Blocker > Fix For: 1.0.0 > > > Current implementation of HoodieSparkParquetReader isn't supporting the case > when "hoodie.datasource.write.drop.partition.columns" is set to true. > In that case partition-path values are expected to be parsed from > partition-path and be injected w/in the File Reader (this is behavior of > Spark's own readers) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[
https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762923#comment-17762923
]
Lin Liu edited comment on HUDI-5807 at 9/8/23 1:39 AM:
---
Recompiled the Jar, and reran the script. But we still see the partition-path
column correctly. I checked the .hoodie/hoodie.properties and confirmed the
propertiy is set to true. All properties are listed as follows.
{code:java}
#Updated at 2023-09-08T01:31:55.789Z
#Thu Sep 07 18:31:55 PDT 2023
hoodie.table.timeline.timezone=LOCAL
hoodie.table.keygenerator.class=org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.table.precombine.field=ts
hoodie.table.version=6
hoodie.database.name=
hoodie.datasource.write.hive_style_partitioning=false
hoodie.table.metadata.partitions.inflight=
hoodie.table.checksum=581566219
hoodie.partition.metafile.use.base.format=false
hoodie.table.cdc.enabled=false
hoodie.archivelog.folder=archived
hoodie.table.name=uuid_test
hoodie.populate.meta.fields=true
hoodie.table.type=COPY_ON_WRITE
hoodie.datasource.write.partitionpath.urlencode=false
hoodie.table.base.file.format=PARQUET
hoodie.datasource.write.drop.partition.columns=true
hoodie.table.metadata.partitions=files
hoodie.timeline.layout.version=1
hoodie.table.recordkey.fields=uuid
hoodie.table.partition.fields=partitionpath {code}
And the query result still shows the partition-path column.
{code:java}
scala> spark.sql("select * from uuid_test_view limit 1").show(false)
+---+-++-++--+--+--+--+--+-+-+-++-+
|_hoodie_commit_time|_hoodie_commit_seqno |_hoodie_record_key
|_hoodie_partition_path |_hoodie_file_name
|begin_lat |begin_lon |driver |end_lat
|end_lon |fare |rider |ts |uuid
|partitionpath |
+---+-++-++--+--+--+--+--+-+-+-++-+
|20230907183152377
|20230907183152377_0_0|0f760d6c-16a7-4d25-9093-440119df0b3f|americas/brazil/sao_paulo|7942e0b7-66fb-4a7a-9883-174a83746814-0_0-24-23_20230907183152377.parquet|0.8946621240010502|0.7425723852244098|driver-213|0.9451772959802982|0.6526397615401383|30.98679973544559|rider-213|1693821831384|0f760d6c-16a7-4d25-9093-440119df0b3f|americas/brazil/sao_paulo|
+---+-++-++--+--+--+--+--+-+-+-++-+
{code}
was (Author: JIRAUSER301185):
Recompiled the Jar, and reran the script. But we still see the partition-path
column correctly. I checked the .hoodie/hoodie.properties and confirmed the
propertiy is set to true. All properties are listed as follows.
{code:java}
#Updated at 2023-09-08T01:31:55.789Z
#Thu Sep 07 18:31:55 PDT 2023
hoodie.table.timeline.timezone=LOCAL
hoodie.table.keygenerator.class=org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.table.precombine.field=ts
hoodie.table.version=6
hoodie.database.name=
hoodie.datasource.write.hive_style_partitioning=false
hoodie.table.metadata.partitions.inflight=
hoodie.table.checksum=581566219
hoodie.partition.metafile.use.base.format=false
hoodie.table.cdc.enabled=false
hoodie.archivelog.folder=archived
hoodie.table.name=uuid_test
hoodie.populate.meta.fields=true
hoodie.table.type=COPY_ON_WRITE
hoodie.datasource.write.partitionpath.urlencode=false
hoodie.table.base.file.format=PARQUET
hoodie.datasource.write.drop.partition.columns=true
hoodie.table.metadata.partitions=files
hoodie.timeline.layout.version=1
hoodie.table.recordkey.fields=uuid
hoodie.table.partition.fields=partitionpath {code}
> HoodieSparkParquetReader is not appending partition-path values
> ---
>
> Key: HUDI-5807
> URL: https://issues.apache.org/jira/browse/HUDI-5807
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark
>Affects Versions: 0.13.0
>Reporter: Alexey Kudinkin
>
[jira] [Commented] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[
https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762923#comment-17762923
]
Lin Liu commented on HUDI-5807:
---
Recompiled the Jar, and reran the script. But we still see the partition-path
column correctly. I checked the .hoodie/hoodie.properties and confirmed the
propertiy is set to true. All properties are listed as follows.
{code:java}
#Updated at 2023-09-08T01:31:55.789Z
#Thu Sep 07 18:31:55 PDT 2023
hoodie.table.timeline.timezone=LOCAL
hoodie.table.keygenerator.class=org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.table.precombine.field=ts
hoodie.table.version=6
hoodie.database.name=
hoodie.datasource.write.hive_style_partitioning=false
hoodie.table.metadata.partitions.inflight=
hoodie.table.checksum=581566219
hoodie.partition.metafile.use.base.format=false
hoodie.table.cdc.enabled=false
hoodie.archivelog.folder=archived
hoodie.table.name=uuid_test
hoodie.populate.meta.fields=true
hoodie.table.type=COPY_ON_WRITE
hoodie.datasource.write.partitionpath.urlencode=false
hoodie.table.base.file.format=PARQUET
hoodie.datasource.write.drop.partition.columns=true
hoodie.table.metadata.partitions=files
hoodie.timeline.layout.version=1
hoodie.table.recordkey.fields=uuid
hoodie.table.partition.fields=partitionpath {code}
> HoodieSparkParquetReader is not appending partition-path values
> ---
>
> Key: HUDI-5807
> URL: https://issues.apache.org/jira/browse/HUDI-5807
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark
>Affects Versions: 0.13.0
>Reporter: Alexey Kudinkin
>Assignee: Lin Liu
>Priority: Blocker
> Fix For: 1.0.0
>
>
> Current implementation of HoodieSparkParquetReader isn't supporting the case
> when "hoodie.datasource.write.drop.partition.columns" is set to true.
> In that case partition-path values are expected to be parsed from
> partition-path and be injected w/in the File Reader (this is behavior of
> Spark's own readers)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[ https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762922#comment-17762922 ] Lin Liu commented on HUDI-5807: --- Checked the hoodie.properties file, and confirmed that hoodie.datasource.write.drop.partition.columns=false. So it means that I did not set it correctly in previous tests. I have modified into the configuration in the hudi code, and set the default value is true. Will retry. > HoodieSparkParquetReader is not appending partition-path values > --- > > Key: HUDI-5807 > URL: https://issues.apache.org/jira/browse/HUDI-5807 > Project: Apache Hudi > Issue Type: Bug > Components: spark >Affects Versions: 0.13.0 >Reporter: Alexey Kudinkin >Assignee: Lin Liu >Priority: Blocker > Fix For: 1.0.0 > > > Current implementation of HoodieSparkParquetReader isn't supporting the case > when "hoodie.datasource.write.drop.partition.columns" is set to true. > In that case partition-path values are expected to be parsed from > partition-path and be injected w/in the File Reader (this is behavior of > Spark's own readers) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] hudi-bot commented on pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
hudi-bot commented on PR #9650: URL: https://github.com/apache/hudi/pull/9650#issuecomment-1710947463 ## CI report: * 634e9882daaf39ee1f5cb8612b5213e3a9bc0f6b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19726) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
danny0405 commented on code in PR #9650: URL: https://github.com/apache/hudi/pull/9650#discussion_r1319245196 ## hudi-gcp/src/main/java/org/apache/hudi/gcp/bigquery/HoodieBigQuerySyncClient.java: ## @@ -94,8 +94,9 @@ public void createTableUsingBqManifestFile(String tableName, String bqManifestFi } String query = String.format( - "CREATE EXTERNAL TABLE `%s.%s` %s OPTIONS (%s " + "CREATE EXTERNAL TABLE `%s.%s.%s` %s OPTIONS (%s " + "uris=[\"%s\"], format=\"PARQUET\", file_set_spec_type=\"NEW_LINE_DELIMITED_MANIFEST\")", + projectId, Review Comment: Nice catch ~ Is the projectId always required for BigQuery sync? Is there any default value for it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
danny0405 commented on code in PR #9650: URL: https://github.com/apache/hudi/pull/9650#discussion_r1319245196 ## hudi-gcp/src/main/java/org/apache/hudi/gcp/bigquery/HoodieBigQuerySyncClient.java: ## @@ -94,8 +94,9 @@ public void createTableUsingBqManifestFile(String tableName, String bqManifestFi } String query = String.format( - "CREATE EXTERNAL TABLE `%s.%s` %s OPTIONS (%s " + "CREATE EXTERNAL TABLE `%s.%s.%s` %s OPTIONS (%s " + "uris=[\"%s\"], format=\"PARQUET\", file_set_spec_type=\"NEW_LINE_DELIMITED_MANIFEST\")", + projectId, Review Comment: Nice catch ~ Is the projectId always required for BigQuery sync? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
danny0405 commented on code in PR #9650: URL: https://github.com/apache/hudi/pull/9650#discussion_r1319245196 ## hudi-gcp/src/main/java/org/apache/hudi/gcp/bigquery/HoodieBigQuerySyncClient.java: ## @@ -94,8 +94,9 @@ public void createTableUsingBqManifestFile(String tableName, String bqManifestFi } String query = String.format( - "CREATE EXTERNAL TABLE `%s.%s` %s OPTIONS (%s " + "CREATE EXTERNAL TABLE `%s.%s.%s` %s OPTIONS (%s " + "uris=[\"%s\"], format=\"PARQUET\", file_set_spec_type=\"NEW_LINE_DELIMITED_MANIFEST\")", + projectId, Review Comment: Nice catch ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 closed pull request #9637: [HUDI-6823] initial writeTimer in StreamWriteOperatorCoordinator
danny0405 closed pull request #9637: [HUDI-6823] initial writeTimer in StreamWriteOperatorCoordinator URL: https://github.com/apache/hudi/pull/9637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9558: [HUDI-6481] Support run multi tables services in a single spark job
danny0405 commented on code in PR #9558:
URL: https://github.com/apache/hudi/pull/9558#discussion_r1317961289
##
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieCleaner.java:
##
@@ -64,6 +64,13 @@ public HoodieCleaner(Config cfg, JavaSparkContext jssc) {
LOG.info("Creating Cleaner with configs : " + props.toString());
}
+ public HoodieCleaner(Config cfg, TypedProperties props, JavaSparkContext
jssc) {
+this.cfg = cfg;
+this.jssc = jssc;
+this.props = props;
Review Comment:
Then we can force one constructor to invoke the other one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #9545: [HUDI-6758] Detecting and skipping Spurious log blocks with MOR reads
danny0405 commented on PR #9545: URL: https://github.com/apache/hudi/pull/9545#issuecomment-1710932980 > Our expectation is readers are upgraded first. It's made clear in every release notes. It's okay if that is already our compatibility routine, in production, readers tends to be im multiple versions. So keeping reading backward compatibility is also important. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[ https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762919#comment-17762919 ] Lin Liu commented on HUDI-5807: --- [~guoyihua], please help check if the configuration is set correctly in my script. > HoodieSparkParquetReader is not appending partition-path values > --- > > Key: HUDI-5807 > URL: https://issues.apache.org/jira/browse/HUDI-5807 > Project: Apache Hudi > Issue Type: Bug > Components: spark >Affects Versions: 0.13.0 >Reporter: Alexey Kudinkin >Assignee: Lin Liu >Priority: Blocker > Fix For: 1.0.0 > > > Current implementation of HoodieSparkParquetReader isn't supporting the case > when "hoodie.datasource.write.drop.partition.columns" is set to true. > In that case partition-path values are expected to be parsed from > partition-path and be injected w/in the File Reader (this is behavior of > Spark's own readers) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[
https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762918#comment-17762918
]
Lin Liu commented on HUDI-5807:
---
Tried to reproduce the issue, and run the following script in local, and the
partition_path column showed. Not sure if this issue has been fixed already. My
script is as follows.
{code:java}
spark-shell --jars
/Users/linliu/projects/hudi/packaging/hudi-spark-bundle/target/hudi-spark3.4-bundle_2.12-1.0.0-SNAPSHOT.jar,/Users/linliu/.m2/repository/com/github/f4b6a3/uuid-creator/5.3.2/uuid-creator-5.3.2.jar
\
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer' \
--conf
'spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog'
\
--conf
'spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension' \
--conf 'hoodie.datasource.write.drop.partition.columns=true'
import org.apache.hudi.QuickstartUtils._
import scala.collection.JavaConversions._
import org.apache.spark.sql.SaveMode._
import org.apache.hudi.DataSourceReadOptions._
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.hudi.common.model.HoodieRecordval tableName = "uuid_test"
val basePath = "file:///tmp/uuid_test1"
val dataGen = new DataGenerator
val inserts = convertToStringList(dataGen.generateInserts(1))
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 1))
df.write.format("hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
mode(Overwrite).
save(basePath)val uuidv_DF = spark.
read.
format("hudi").
load(basePath)
uuidv_DF.createOrReplaceTempView("uuid_test_view")spark.sql("select * from
uuid_test_view limit 10").show(false)
{code}
The result is as follows.
{code:java}
scala> spark.sql("select * from uuid_test_view limit 1").show(false)
+---+-++-++-+---+--++---+-+-+-++-+
|_hoodie_commit_time|_hoodie_commit_seqno |_hoodie_record_key
|_hoodie_partition_path |_hoodie_file_name
|begin_lat |begin_lon |driver |end_lat
|end_lon |fare |rider |ts |uuid
|partitionpath |
+---+-++-++-+---+--++---+-+-+-++-+
|20230907173941752
|20230907173941752_0_0|5911f8ea-94e5-4803-b3c4-aa574d8bab06|americas/brazil/sao_paulo|ea97ba95-dcfd-43f0-953b-9aac94a70474-0_0-24-23_20230907173941752.parquet|0.864670567383501|0.07253990487069728|driver-213|0.024148074339951364|0.12390571197157785|49.37822483824809|rider-213|1693834352147|5911f8ea-94e5-4803-b3c4-aa574d8bab06|americas/brazil/sao_paulo|
+---+-++-++-+---+--++---+-+-+-++-+{code}
> HoodieSparkParquetReader is not appending partition-path values
> ---
>
> Key: HUDI-5807
> URL: https://issues.apache.org/jira/browse/HUDI-5807
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark
>Affects Versions: 0.13.0
>Reporter: Alexey Kudinkin
>Assignee: Lin Liu
>Priority: Blocker
> Fix For: 1.0.0
>
>
> Current implementation of HoodieSparkParquetReader isn't supporting the case
> when "hoodie.datasource.write.drop.partition.columns" is set to true.
> In that case partition-path values are expected to be parsed from
> partition-path and be injected w/in the File Reader (this is behavior of
> Spark's own readers)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HUDI-6831) Add back missing project_id to query statement in BigQuerySyncTool
[ https://issues.apache.org/jira/browse/HUDI-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinpeng Zhou updated HUDI-6831: --- Description: Currently the ddl using bq manifest file is "create external table dataset.table" which does not include the project id. It should be "project.dataset.table" (was: The project id is missing in the query statement when using bigquery manifest file. Currently it's "create external table dataset.table" and does not work for cross-project scenarios. It should be fixed to "create external table project.dataset.table") > Add back missing project_id to query statement in BigQuerySyncTool > -- > > Key: HUDI-6831 > URL: https://issues.apache.org/jira/browse/HUDI-6831 > Project: Apache Hudi > Issue Type: Bug > Components: meta-sync >Reporter: Jinpeng Zhou >Priority: Minor > Fix For: 0.14.0 > > > Currently the ddl using bq manifest file is "create external table > dataset.table" which does not include the project id. It should be > "project.dataset.table" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] hudi-bot commented on pull request #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
hudi-bot commented on PR #9650: URL: https://github.com/apache/hudi/pull/9650#issuecomment-1710921429 ## CI report: * 634e9882daaf39ee1f5cb8612b5213e3a9bc0f6b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9606: [HUDI-6820] Test CI Timeout(0.13.0) - 3
hudi-bot commented on PR #9606: URL: https://github.com/apache/hudi/pull/9606#issuecomment-1710916073 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * bed300bac585296a16e1cb2570b84a0538f1bcd1 UNKNOWN * 45a71704ba1dae283aa199134fbd72a80da483b5 UNKNOWN * ec7588d4ccd708f47d6cc3e781560011fac807d1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19711) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9609: [HUDI-6820] Test CI Timeout(0.13.0) - 6
hudi-bot commented on PR #9609: URL: https://github.com/apache/hudi/pull/9609#issuecomment-1710916204 ## CI report: * e82bbacdd446180d5846ff46f528541c5b48fada UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 0156a57e3dfa545c800706e03f97faab3969375e UNKNOWN * 982ea6df4b96d844a97c415b5d6dbed3ea44346a UNKNOWN * e2a58e8dab1cc9e39b9536bfa12bcf348720dbd4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19714) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9633: [HUDI-6820] Test CI Timeout(0.13.0) - 8
hudi-bot commented on PR #9633: URL: https://github.com/apache/hudi/pull/9633#issuecomment-1710916323 ## CI report: * e556ee21fb3dc2a13596e92a9ead07ba2d0be0fd UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 7b9943781a2c42e6c19d0b2d2ade8febfceffbcb UNKNOWN * 0fa022808aae57dec7414e70cbb5cf4188f4673b Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19716) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9632: [HUDI-6820] Test CI Timeout(0.13.0) - 7
hudi-bot commented on PR #9632: URL: https://github.com/apache/hudi/pull/9632#issuecomment-1710916293 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 47b820626163858872b6a3e7615c968864012af7 UNKNOWN * 729906cc06ca44c4137ee95406f3248c7904ade0 UNKNOWN * f057b6cd9c7db1da8d3e3391866dc1e1ce8751e2 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19715) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9608: [HUDI-6820] Test CI Timeout(0.13.0) - 5
hudi-bot commented on PR #9608: URL: https://github.com/apache/hudi/pull/9608#issuecomment-1710916171 ## CI report: * 4f3531c739db21d3a5b46f343342b535226dce16 UNKNOWN * 459d98262590cd0ca0af0fc54578e5ac413afe0c UNKNOWN * 17dfaaa989b2e52861e3319f03ac058bbe9e2c39 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19713) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9607: [HUDI-6820] Test CI Timeout(0.13.0) - 4
hudi-bot commented on PR #9607: URL: https://github.com/apache/hudi/pull/9607#issuecomment-1710916121 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * a628366cf7ce6d091d59e822654da602b7e6e49d UNKNOWN * d56c7e7d88071966ff6a6b65ddc919bf595ff96b UNKNOWN * 5c5b9724c7aed3835b34987c2f69ae47fab729ca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19712) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] jp0317 opened a new pull request, #9650: [HUDI-6831] Add back missing project_id to query statement in BigQuerySyncTool
jp0317 opened a new pull request, #9650: URL: https://github.com/apache/hudi/pull/9650 ### Change Logs The project id is missing in the query statement when using bigquery manifest file. Currently it's "create external table dataset.table" and does not work for cross-project scenarios. It should be fixed to "create external table project.dataset.table" ### Impact Fix the bug which failed cross-project scenarios. ### Risk level (write none, low medium or high below) None ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HUDI-6831) Add back missing project_id to query statement in BigQuerySyncTool
Jinpeng Zhou created HUDI-6831: -- Summary: Add back missing project_id to query statement in BigQuerySyncTool Key: HUDI-6831 URL: https://issues.apache.org/jira/browse/HUDI-6831 Project: Apache Hudi Issue Type: Bug Components: meta-sync Reporter: Jinpeng Zhou Fix For: 0.14.0 The project id is missing in the query statement when using bigquery manifest file. Currently it's "create external table dataset.table" and does not work for cross-project scenarios. It should be fixed to "create external table project.dataset.table" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-5807) HoodieSparkParquetReader is not appending partition-path values
[ https://issues.apache.org/jira/browse/HUDI-5807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu updated HUDI-5807: -- Status: In Progress (was: Open) > HoodieSparkParquetReader is not appending partition-path values > --- > > Key: HUDI-5807 > URL: https://issues.apache.org/jira/browse/HUDI-5807 > Project: Apache Hudi > Issue Type: Bug > Components: spark >Affects Versions: 0.13.0 >Reporter: Alexey Kudinkin >Assignee: Lin Liu >Priority: Blocker > Fix For: 1.0.0 > > > Current implementation of HoodieSparkParquetReader isn't supporting the case > when "hoodie.datasource.write.drop.partition.columns" is set to true. > In that case partition-path values are expected to be parsed from > partition-path and be injected w/in the File Reader (this is behavior of > Spark's own readers) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HUDI-6765) Add merge mode to allow differentiation of dedup logic
[ https://issues.apache.org/jira/browse/HUDI-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Liu reassigned HUDI-6765: - Assignee: Lin Liu > Add merge mode to allow differentiation of dedup logic > -- > > Key: HUDI-6765 > URL: https://issues.apache.org/jira/browse/HUDI-6765 > Project: Apache Hudi > Issue Type: Task >Reporter: Ethan Guo >Assignee: Lin Liu >Priority: Blocker > Fix For: 1.0.0 > > > The new merge API in HoodieRecordMerger can only differentiate merge logic > based on `OperationModeAwareness`, which introduces another layer of > complexity. Instead, could we add a new argument of merge mode (pre-combine, > or update) to the merge API for customized dedup (or merging of log > records?), instead of using OperationModeAwareness? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] hudi-bot commented on pull request #9649: [HUDI-6826] Port BloomFilter related classes from Hadoop library
hudi-bot commented on PR #9649: URL: https://github.com/apache/hudi/pull/9649#issuecomment-1710878014 ## CI report: * 3d62171eaeb3cc64a5878462d9bc0606b571770b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19725) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9605: [HUDI-6820] Test CI Timeout(0.13.0) - 2
hudi-bot commented on PR #9605: URL: https://github.com/apache/hudi/pull/9605#issuecomment-1710877741 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 3b2a6c639d395b03ff6c8cb8b99317f6ddda7547 UNKNOWN * 957d6f6afb74298902a0f322f4e90459372062fe Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19710) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9649: [HUDI-6826] Port BloomFilter related classes from Hadoop library
hudi-bot commented on PR #9649: URL: https://github.com/apache/hudi/pull/9649#issuecomment-1710843346 ## CI report: * 3d62171eaeb3cc64a5878462d9bc0606b571770b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9648: [HUDI-6820] Triage ci timeouts(3)
hudi-bot commented on PR #9648: URL: https://github.com/apache/hudi/pull/9648#issuecomment-1710843311 ## CI report: * e2574f456376f0be5a7ed774f5468f795b3bdcf4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19724) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9647: [HUDI-6820] Triage ci timeouts(2)
hudi-bot commented on PR #9647: URL: https://github.com/apache/hudi/pull/9647#issuecomment-1710843288 ## CI report: * 68604dbdfa581a6e828dbd7a62fdc3f4417384b9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19723) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9646: [HUDI-6820] Running CI to triage time outs (1)
hudi-bot commented on PR #9646: URL: https://github.com/apache/hudi/pull/9646#issuecomment-1710843253 ## CI report: * 847f94e17ff7a93d133056f75db59daf54fd45a4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19722) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9645: [HUDI-6820] Writer client close(3)
hudi-bot commented on PR #9645: URL: https://github.com/apache/hudi/pull/9645#issuecomment-1710836554 ## CI report: * 1aa4f82d4a4640371d8e62ed76599897d1a2c8dc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19721) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19720) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9604: [HUDI-6820] Test CI Timeout(0.13.0) - 1
hudi-bot commented on PR #9604: URL: https://github.com/apache/hudi/pull/9604#issuecomment-1710836254 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 736bf3316006fbe1211e3693b3b0e75390930408 UNKNOWN * ba2074392eea11e003e551f5fff84bd566ae0168 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19709) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9648: [HUDI-6820] Triage ci timeouts(3)
hudi-bot commented on PR #9648: URL: https://github.com/apache/hudi/pull/9648#issuecomment-1710836647 ## CI report: * e2574f456376f0be5a7ed774f5468f795b3bdcf4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9646: [HUDI-6820] Running CI to triage time outs (1)
hudi-bot commented on PR #9646: URL: https://github.com/apache/hudi/pull/9646#issuecomment-1710836579 ## CI report: * 847f94e17ff7a93d133056f75db59daf54fd45a4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9647: [HUDI-6820] Triage ci timeouts(2)
hudi-bot commented on PR #9647: URL: https://github.com/apache/hudi/pull/9647#issuecomment-1710836610 ## CI report: * 68604dbdfa581a6e828dbd7a62fdc3f4417384b9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua opened a new pull request, #9649: [HUDI-6826] Port BloomFilter related classes from Hadoop library
yihua opened a new pull request, #9649: URL: https://github.com/apache/hudi/pull/9649 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HUDI-6826) Port BloomFilter related classes from Hadoop library to remove dependency
[ https://issues.apache.org/jira/browse/HUDI-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6826: Summary: Port BloomFilter related classes from Hadoop library to remove dependency (was: Copy BloomFilter class from Hadoop library to remove dependency) > Port BloomFilter related classes from Hadoop library to remove dependency > - > > Key: HUDI-6826 > URL: https://issues.apache.org/jira/browse/HUDI-6826 > Project: Apache Hudi > Issue Type: Task >Reporter: Ethan Guo >Assignee: Ethan Guo >Priority: Blocker > Fix For: 1.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-6826) Copy BloomFilter class from Hadoop library to remove dependency
[ https://issues.apache.org/jira/browse/HUDI-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6826: Story Points: 4 (was: 0.5) > Copy BloomFilter class from Hadoop library to remove dependency > --- > > Key: HUDI-6826 > URL: https://issues.apache.org/jira/browse/HUDI-6826 > Project: Apache Hudi > Issue Type: Task >Reporter: Ethan Guo >Assignee: Ethan Guo >Priority: Blocker > Fix For: 1.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HUDI-6825) Use UTF_8 to encode String to byte array in all places
[ https://issues.apache.org/jira/browse/HUDI-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6825: Story Points: 1 (was: 2) > Use UTF_8 to encode String to byte array in all places > -- > > Key: HUDI-6825 > URL: https://issues.apache.org/jira/browse/HUDI-6825 > Project: Apache Hudi > Issue Type: Task >Reporter: Ethan Guo >Assignee: Ethan Guo >Priority: Blocker > Fix For: 1.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hudi] hudi-bot commented on pull request #9645: [HUDI-6820] Writer client close(3)
hudi-bot commented on PR #9645: URL: https://github.com/apache/hudi/pull/9645#issuecomment-1710800951 ## CI report: * 1aa4f82d4a4640371d8e62ed76599897d1a2c8dc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19720) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19721) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan opened a new pull request, #9648: [HUDI-6820] Triage ci timeouts(3)
nsivabalan opened a new pull request, #9648: URL: https://github.com/apache/hudi/pull/9648 ### Change Logs Triage ci timeouts(3) ### Impact Triage ci timeouts(3) ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #9645: [HUDI-6820] Writer client close(3)
nsivabalan commented on PR #9645: URL: https://github.com/apache/hudi/pull/9645#issuecomment-1710797221 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan opened a new pull request, #9647: [HUDI-6820] Triage ci timeouts(2)
nsivabalan opened a new pull request, #9647: URL: https://github.com/apache/hudi/pull/9647 ### Change Logs Triage ci timeouts(2) ### Impact Triage ci timeouts(2) ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan opened a new pull request, #9646: [HUDI-6820] Running CI to triage time outs (1)
nsivabalan opened a new pull request, #9646: URL: https://github.com/apache/hudi/pull/9646 ### Change Logs Running CI to triage time outs (1) ### Impact Running CI to triage time outs (1) ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] bhasudha commented on pull request #9622: [WIP] Fixing quick start guide for pkless
bhasudha commented on PR #9622: URL: https://github.com/apache/hudi/pull/9622#issuecomment-1710792732 Reviewed the flow with @nsivabalan . looks good overall. Will take a look once after the full pr is up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9645: [HUDI-6820] Writer client close(3)
hudi-bot commented on PR #9645: URL: https://github.com/apache/hudi/pull/9645#issuecomment-1710792634 ## CI report: * 1aa4f82d4a4640371d8e62ed76599897d1a2c8dc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9644: [HUDI-6820] Closing Write clients (2)
hudi-bot commented on PR #9644: URL: https://github.com/apache/hudi/pull/9644#issuecomment-1710784264 ## CI report: * ef1e29705531b56996e35b1b9137ac9318dc1677 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19719) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9637: [HUDI-6823] initial writeTimer in StreamWriteOperatorCoordinator
hudi-bot commented on PR #9637: URL: https://github.com/apache/hudi/pull/9637#issuecomment-1710784168 ## CI report: * a6f3e3e4397627c612e28b12000b43b96a1d4a26 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19699) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19705) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9643: [HUDI-6820] Write client closure in tests (1)
hudi-bot commented on PR #9643: URL: https://github.com/apache/hudi/pull/9643#issuecomment-1710784237 ## CI report: * bb8f4bce70af2ec873dbfefea43a4a04c55307cf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19718) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9644: [HUDI-6820] Closing Write clients (2)
hudi-bot commented on PR #9644: URL: https://github.com/apache/hudi/pull/9644#issuecomment-1710737796 ## CI report: * ef1e29705531b56996e35b1b9137ac9318dc1677 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan opened a new pull request, #9645: [HUDI-6820] Writer client close(3)
nsivabalan opened a new pull request, #9645: URL: https://github.com/apache/hudi/pull/9645 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9643: [HUDI-6820] Write client closure in tests (1)
hudi-bot commented on PR #9643: URL: https://github.com/apache/hudi/pull/9643#issuecomment-1710737754 ## CI report: * bb8f4bce70af2ec873dbfefea43a4a04c55307cf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9642: [HUDI-6820] Close write clients in tests
hudi-bot commented on PR #9642: URL: https://github.com/apache/hudi/pull/9642#issuecomment-1710737709 ## CI report: * c4e998719c4af415efad6d5aeb84e5705c3761c0 UNKNOWN * fe6adfb876c41ab5ed8c261e661f9dfdf147826a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19704) * bb8f4bce70af2ec873dbfefea43a4a04c55307cf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19717) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9642: [HUDI-6820] Close write clients in tests
hudi-bot commented on PR #9642: URL: https://github.com/apache/hudi/pull/9642#issuecomment-1710728064 ## CI report: * c4e998719c4af415efad6d5aeb84e5705c3761c0 UNKNOWN * fe6adfb876c41ab5ed8c261e661f9dfdf147826a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19704) * bb8f4bce70af2ec873dbfefea43a4a04c55307cf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan opened a new pull request, #9644: [HUDI-6820] Closing Write clients (2)
nsivabalan opened a new pull request, #9644: URL: https://github.com/apache/hudi/pull/9644 ### Change Logs Closing Write clients (2) ### Impact Stable CI ### Risk level (write none, low medium or high below) low ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan opened a new pull request, #9643: [HUDI-6820] Write client closure in tests (1)
nsivabalan opened a new pull request, #9643: URL: https://github.com/apache/hudi/pull/9643 ### Change Logs Write client closure in tests (1) ### Impact Write client closure in tests (1) ### Risk level (write none, low medium or high below) low ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] dht7 closed issue #9042: [SUPPORT] Cannot write nullable values to non-null column
dht7 closed issue #9042: [SUPPORT] Cannot write nullable values to non-null column URL: https://github.com/apache/hudi/issues/9042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9633: [HUDI-6820] Test CI Timeout(0.13.0) - 8
hudi-bot commented on PR #9633: URL: https://github.com/apache/hudi/pull/9633#issuecomment-1710717176 ## CI report: * e556ee21fb3dc2a13596e92a9ead07ba2d0be0fd UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 7b9943781a2c42e6c19d0b2d2ade8febfceffbcb UNKNOWN * 0fa022808aae57dec7414e70cbb5cf4188f4673b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19716) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9632: [HUDI-6820] Test CI Timeout(0.13.0) - 7
hudi-bot commented on PR #9632: URL: https://github.com/apache/hudi/pull/9632#issuecomment-1710717114 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 47b820626163858872b6a3e7615c968864012af7 UNKNOWN * 729906cc06ca44c4137ee95406f3248c7904ade0 UNKNOWN * f057b6cd9c7db1da8d3e3391866dc1e1ce8751e2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19715) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9609: [HUDI-6820] Test CI Timeout(0.13.0) - 6
hudi-bot commented on PR #9609: URL: https://github.com/apache/hudi/pull/9609#issuecomment-1710716965 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19629) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19627) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * e82bbacdd446180d5846ff46f528541c5b48fada UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 0156a57e3dfa545c800706e03f97faab3969375e UNKNOWN * 982ea6df4b96d844a97c415b5d6dbed3ea44346a UNKNOWN * e2a58e8dab1cc9e39b9536bfa12bcf348720dbd4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19714) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9608: [HUDI-6820] Test CI Timeout(0.13.0) - 5
hudi-bot commented on PR #9608: URL: https://github.com/apache/hudi/pull/9608#issuecomment-1710716901 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * 4f3531c739db21d3a5b46f343342b535226dce16 UNKNOWN * 459d98262590cd0ca0af0fc54578e5ac413afe0c UNKNOWN * 17dfaaa989b2e52861e3319f03ac058bbe9e2c39 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19713) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9607: [HUDI-6820] Test CI Timeout(0.13.0) - 4
hudi-bot commented on PR #9607: URL: https://github.com/apache/hudi/pull/9607#issuecomment-1710716840 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * a628366cf7ce6d091d59e822654da602b7e6e49d UNKNOWN * d56c7e7d88071966ff6a6b65ddc919bf595ff96b UNKNOWN * 5c5b9724c7aed3835b34987c2f69ae47fab729ca Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19712) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9606: [HUDI-6820] Test CI Timeout(0.13.0) - 3
hudi-bot commented on PR #9606: URL: https://github.com/apache/hudi/pull/9606#issuecomment-1710716784 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19627) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * bed300bac585296a16e1cb2570b84a0538f1bcd1 UNKNOWN * 45a71704ba1dae283aa199134fbd72a80da483b5 UNKNOWN * ec7588d4ccd708f47d6cc3e781560011fac807d1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19711) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9605: [HUDI-6820] Test CI Timeout(0.13.0) - 2
hudi-bot commented on PR #9605: URL: https://github.com/apache/hudi/pull/9605#issuecomment-1710716680 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 3b2a6c639d395b03ff6c8cb8b99317f6ddda7547 UNKNOWN * 26779bca0ba439abc02459ee7a4e4f6761af0122 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19707) * 957d6f6afb74298902a0f322f4e90459372062fe Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19710) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9604: [HUDI-6820] Test CI Timeout(0.13.0) - 1
hudi-bot commented on PR #9604: URL: https://github.com/apache/hudi/pull/9604#issuecomment-1710716538 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 736bf3316006fbe1211e3693b3b0e75390930408 UNKNOWN * c1ba9bba28ff37443b40c2ceafade7213aaee13f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19706) * ba2074392eea11e003e551f5fff84bd566ae0168 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19709) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9633: [HUDI-6820] Test CI Timeout(0.13.0) - 8
hudi-bot commented on PR #9633: URL: https://github.com/apache/hudi/pull/9633#issuecomment-1710669489 ## CI report: * e556ee21fb3dc2a13596e92a9ead07ba2d0be0fd UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 7b9943781a2c42e6c19d0b2d2ade8febfceffbcb UNKNOWN * 0fa022808aae57dec7414e70cbb5cf4188f4673b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9632: [HUDI-6820] Test CI Timeout(0.13.0) - 7
hudi-bot commented on PR #9632: URL: https://github.com/apache/hudi/pull/9632#issuecomment-1710669440 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 47b820626163858872b6a3e7615c968864012af7 UNKNOWN * 729906cc06ca44c4137ee95406f3248c7904ade0 UNKNOWN * f057b6cd9c7db1da8d3e3391866dc1e1ce8751e2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9609: [HUDI-6820] Test CI Timeout(0.13.0) - 6
hudi-bot commented on PR #9609: URL: https://github.com/apache/hudi/pull/9609#issuecomment-1710669253 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19629) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19627) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * e82bbacdd446180d5846ff46f528541c5b48fada UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 0156a57e3dfa545c800706e03f97faab3969375e UNKNOWN * 982ea6df4b96d844a97c415b5d6dbed3ea44346a UNKNOWN * e2a58e8dab1cc9e39b9536bfa12bcf348720dbd4 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9608: [HUDI-6820] Test CI Timeout(0.13.0) - 5
hudi-bot commented on PR #9608: URL: https://github.com/apache/hudi/pull/9608#issuecomment-1710669200 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * 4f3531c739db21d3a5b46f343342b535226dce16 UNKNOWN * 459d98262590cd0ca0af0fc54578e5ac413afe0c UNKNOWN * 17dfaaa989b2e52861e3319f03ac058bbe9e2c39 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9607: [HUDI-6820] Test CI Timeout(0.13.0) - 4
hudi-bot commented on PR #9607: URL: https://github.com/apache/hudi/pull/9607#issuecomment-1710669160 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * a628366cf7ce6d091d59e822654da602b7e6e49d UNKNOWN * d56c7e7d88071966ff6a6b65ddc919bf595ff96b UNKNOWN * 5c5b9724c7aed3835b34987c2f69ae47fab729ca UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9605: [HUDI-6820] Test CI Timeout(0.13.0) - 2
hudi-bot commented on PR #9605: URL: https://github.com/apache/hudi/pull/9605#issuecomment-1710669056 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19629) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 3b2a6c639d395b03ff6c8cb8b99317f6ddda7547 UNKNOWN * 26779bca0ba439abc02459ee7a4e4f6761af0122 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19707) * 957d6f6afb74298902a0f322f4e90459372062fe UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9606: [HUDI-6820] Test CI Timeout(0.13.0) - 3
hudi-bot commented on PR #9606: URL: https://github.com/apache/hudi/pull/9606#issuecomment-1710669114 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19627) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * bed300bac585296a16e1cb2570b84a0538f1bcd1 UNKNOWN * 45a71704ba1dae283aa199134fbd72a80da483b5 UNKNOWN * ec7588d4ccd708f47d6cc3e781560011fac807d1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9604: [HUDI-6820] Test CI Timeout(0.13.0) - 1
hudi-bot commented on PR #9604: URL: https://github.com/apache/hudi/pull/9604#issuecomment-1710668992 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19629) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19627) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19696) * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 736bf3316006fbe1211e3693b3b0e75390930408 UNKNOWN * c1ba9bba28ff37443b40c2ceafade7213aaee13f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19706) * ba2074392eea11e003e551f5fff84bd566ae0168 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9633: [HUDI-6820] Test CI Timeout(0.13.0) - 8
hudi-bot commented on PR #9633: URL: https://github.com/apache/hudi/pull/9633#issuecomment-1710658666 ## CI report: * e556ee21fb3dc2a13596e92a9ead07ba2d0be0fd UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 7b9943781a2c42e6c19d0b2d2ade8febfceffbcb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9632: [HUDI-6820] Test CI Timeout(0.13.0) - 7
hudi-bot commented on PR #9632: URL: https://github.com/apache/hudi/pull/9632#issuecomment-1710658622 ## CI report: * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 47b820626163858872b6a3e7615c968864012af7 UNKNOWN * 729906cc06ca44c4137ee95406f3248c7904ade0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9609: [HUDI-6820] Test CI Timeout(0.13.0) - 6
hudi-bot commented on PR #9609: URL: https://github.com/apache/hudi/pull/9609#issuecomment-1710658466 ## CI report: * 4a0d42c2cb75170afc6b752f23d934ea0e5a2eee Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19629) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19631) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19625) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19624) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19627) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19630) * e82bbacdd446180d5846ff46f528541c5b48fada UNKNOWN * abf8f45b880a7d575c9e6ec17a8b756906ecf70a UNKNOWN * 0156a57e3dfa545c800706e03f97faab3969375e UNKNOWN * 982ea6df4b96d844a97c415b5d6dbed3ea44346a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
