Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
danny0405 merged PR #10101: URL: https://github.com/apache/hudi/pull/10101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1816148208 ## CI report: * 29b2f98a40bd73d7cde23f8cd845fec1c5c22dd1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20969) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1815846194 ## CI report: * 3a4f9e92cd71dac2506c883c57785f07ee5bcf24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20964) * 29b2f98a40bd73d7cde23f8cd845fec1c5c22dd1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20969) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1815839107 ## CI report: * 3a4f9e92cd71dac2506c883c57785f07ee5bcf24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20964) * 29b2f98a40bd73d7cde23f8cd845fec1c5c22dd1 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
danny0405 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396767580
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -111,13 +114,13 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
};
this.timelineWriter.write(instantsToArchive, Option.of(action ->
deleteAnyLeftOverMarkers(context, action)), Option.of(exceptionHandler));
LOG.info("Deleting archived instants " + instantsToArchive);
-success = deleteArchivedInstants(instantsToArchive, context);
+deleteArchivedInstants(instantsToArchive, context);
// triggers compaction and cleaning only after archiving action
this.timelineWriter.compactAndClean(context);
} else {
LOG.info("No Instants to archive");
}
- return success;
+ return instantsToArchive.size();
Review Comment:
We can address it in another PR, we kind of do not have atomicity for
current 2 steps:
- flush of archived timeline
- deletion of active metadata files
My roughly thought is we can fix the left over active metadata files (should
be deleted from active timeline) in the next round of archiving, imagine the
latest instant time in archived timeline is t10 and the oldest instant in
active timeline is t7, we should retry the deletion of instant metadat files
from t7 ~ 10 at the very beginning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
danny0405 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396767580
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -111,13 +114,13 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
};
this.timelineWriter.write(instantsToArchive, Option.of(action ->
deleteAnyLeftOverMarkers(context, action)), Option.of(exceptionHandler));
LOG.info("Deleting archived instants " + instantsToArchive);
-success = deleteArchivedInstants(instantsToArchive, context);
+deleteArchivedInstants(instantsToArchive, context);
// triggers compaction and cleaning only after archiving action
this.timelineWriter.compactAndClean(context);
} else {
LOG.info("No Instants to archive");
}
- return success;
+ return instantsToArchive.size();
Review Comment:
We can address it in another PR, we kind of do not have atomicity for
current 2 steps:
- flush of archived timeline
- deletion of active metadata files
My roughly thought about it, we can fix the left over active metadata files
(should be deleted from active timeline) in the next round of archiving,
imagine the latest instant time in archived timeline is t10 and the oldest
instant in active timeline is t7, we should retry the deletion of instant
metadat files from t7 ~ 10 at the very beginning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1815708138 ## CI report: * 3a4f9e92cd71dac2506c883c57785f07ee5bcf24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20964) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
majian1998 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396670883
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -111,13 +114,13 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
};
this.timelineWriter.write(instantsToArchive, Option.of(action ->
deleteAnyLeftOverMarkers(context, action)), Option.of(exceptionHandler));
LOG.info("Deleting archived instants " + instantsToArchive);
-success = deleteArchivedInstants(instantsToArchive, context);
+deleteArchivedInstants(instantsToArchive, context);
// triggers compaction and cleaning only after archiving action
this.timelineWriter.compactAndClean(context);
} else {
LOG.info("No Instants to archive");
}
- return success;
+ return instantsToArchive.size();
Review Comment:
In the current archive code, it seems that `success` is always set to true.
The `success` variable is initialized as true, and `deleteArchivedInstants`
always returns true unless it fails. However, if there is a failure, the
current implementation should throw an exception and terminate without
returning any value. Therefore, I believe the `success` variable is meaningless
in the current context. Alternatively, should we catch these exceptions and
return false? I'm not sure if this would be reasonable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
danny0405 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396663741
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -111,13 +114,13 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
};
this.timelineWriter.write(instantsToArchive, Option.of(action ->
deleteAnyLeftOverMarkers(context, action)), Option.of(exceptionHandler));
LOG.info("Deleting archived instants " + instantsToArchive);
-success = deleteArchivedInstants(instantsToArchive, context);
+deleteArchivedInstants(instantsToArchive, context);
// triggers compaction and cleaning only after archiving action
this.timelineWriter.compactAndClean(context);
} else {
LOG.info("No Instants to archive");
}
- return success;
+ return instantsToArchive.size();
Review Comment:
Should we also include the `success` flag also as a metric?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1815676361 ## CI report: * 178ef4eadac6ab6d009d86ab86d35babe952 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20942) * 3a4f9e92cd71dac2506c883c57785f07ee5bcf24 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20964) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
stream2000 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396632832
##
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/io/TestHoodieTimelineArchiver.java:
##
@@ -256,8 +256,8 @@ public void testArchiveEmptyTable() throws Exception {
metaClient = HoodieTableMetaClient.reload(metaClient);
HoodieTable table = HoodieSparkTable.create(cfg, context, metaClient);
HoodieTimelineArchiver archiver = new HoodieTimelineArchiver(cfg, table);
-boolean result = archiver.archiveIfRequired(context);
-assertTrue(result);
+int result = archiver.archiveIfRequired(context);
Review Comment:
We can add some tests for archive metrics like what we done for compaction
metrics in #8759.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1815670631 ## CI report: * 178ef4eadac6ab6d009d86ab86d35babe952 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20942) * 3a4f9e92cd71dac2506c883c57785f07ee5bcf24 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
majian1998 commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1815653571 Now, the archive metrics have been moved to `BaseHoodieTableServiceClient`, and the return value of archiveIfRequired has been modified. In the previous implementation, the `success` variable always seemed to be true? So, I have also removed the relevant checks in the unit tests. cc@danny0405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
danny0405 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1396571766
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -117,6 +122,10 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
} else {
LOG.info("No Instants to archive");
}
+ if (success && timerContext != null) {
+long durationMs = metrics.getDurationInMs(timerContext.stop());
Review Comment:
Yeah, need to think through what the return type should look like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
majian1998 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1395620121
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -117,6 +122,10 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
} else {
LOG.info("No Instants to archive");
}
+ if (success && timerContext != null) {
+long durationMs = metrics.getDurationInMs(timerContext.stop());
Review Comment:
This does indeed make the style more consistent, but there is one issue –
the archive action does not return any metadata, so we are unable to obtain
information about the cleaned instants. Do we need to modify the return content
of archiveIfRequired to support this improvement?
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -117,6 +122,10 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
} else {
LOG.info("No Instants to archive");
}
+ if (success && timerContext != null) {
+long durationMs = metrics.getDurationInMs(timerContext.stop());
Review Comment:
This does indeed make the style more consistent, but there is one issue –
the archive action does not return any metadata, so we are unable to obtain
information about the cleaned instants. Do we need to modify the return content
of archiveIfRequired to support this improvement?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1813945486 ## CI report: * 178ef4eadac6ab6d009d86ab86d35babe952 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20942) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
danny0405 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1395270648
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/HoodieTimelineArchiver.java:
##
@@ -117,6 +122,10 @@ public boolean archiveIfRequired(HoodieEngineContext
context, boolean acquireLoc
} else {
LOG.info("No Instants to archive");
}
+ if (success && timerContext != null) {
+long durationMs = metrics.getDurationInMs(timerContext.stop());
Review Comment:
Can we move the metrics handling to the write client or the service client,
the cleaning and rollback alreay follow this pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1813839813 ## CI report: * 2f97634b8b59e9f61dc05b649e78f9fe747c5ee5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20922) * 178ef4eadac6ab6d009d86ab86d35babe952 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20942) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1813833815 ## CI report: * 2f97634b8b59e9f61dc05b649e78f9fe747c5ee5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20922) * 178ef4eadac6ab6d009d86ab86d35babe952 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
stream2000 commented on code in PR #10101:
URL: https://github.com/apache/hudi/pull/10101#discussion_r1395063596
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/HoodieMetrics.java:
##
@@ -255,48 +277,57 @@ private void updateCommitTimingMetrics(long
commitEpochTimeInMs, long durationIn
Pair, Option> eventTimePairMinMax =
metadata.getMinAndMaxEventTime();
if (eventTimePairMinMax.getLeft().isPresent()) {
long commitLatencyInMs = commitEpochTimeInMs + durationInMs -
eventTimePairMinMax.getLeft().get();
-metrics.registerGauge(getMetricsName(actionType, "commitLatencyInMs"),
commitLatencyInMs);
+metrics.registerGauge(getMetricsName(actionType, COMMIT_LATENCY_STR),
commitLatencyInMs);
}
if (eventTimePairMinMax.getRight().isPresent()) {
long commitFreshnessInMs = commitEpochTimeInMs + durationInMs -
eventTimePairMinMax.getRight().get();
-metrics.registerGauge(getMetricsName(actionType,
"commitFreshnessInMs"), commitFreshnessInMs);
+metrics.registerGauge(getMetricsName(actionType,
COMMIT_FRESHNESS_STR), commitFreshnessInMs);
}
- metrics.registerGauge(getMetricsName(actionType, "commitTime"),
commitEpochTimeInMs);
- metrics.registerGauge(getMetricsName(actionType, "duration"),
durationInMs);
+ metrics.registerGauge(getMetricsName(actionType, COMMIT_TIME_STR),
commitEpochTimeInMs);
+ metrics.registerGauge(getMetricsName(actionType, DURATION_STR),
durationInMs);
}
}
public void updateRollbackMetrics(long durationInMs, long numFilesDeleted) {
if (config.isMetricsOn()) {
LOG.info(
String.format("Sending rollback metrics (duration=%d,
numFilesDeleted=%d)", durationInMs, numFilesDeleted));
- metrics.registerGauge(getMetricsName("rollback", "duration"),
durationInMs);
- metrics.registerGauge(getMetricsName("rollback", "numFilesDeleted"),
numFilesDeleted);
+ metrics.registerGauge(getMetricsName(HoodieTimeline.ROLLBACK_ACTION,
DURATION_STR), durationInMs);
+ metrics.registerGauge(getMetricsName(HoodieTimeline.ROLLBACK_ACTION,
DELETE_FILES_NUM_STR), numFilesDeleted);
}
}
public void updateCleanMetrics(long durationInMs, int numFilesDeleted) {
if (config.isMetricsOn()) {
LOG.info(
String.format("Sending clean metrics (duration=%d,
numFilesDeleted=%d)", durationInMs, numFilesDeleted));
- metrics.registerGauge(getMetricsName("clean", "duration"), durationInMs);
- metrics.registerGauge(getMetricsName("clean", "numFilesDeleted"),
numFilesDeleted);
+ metrics.registerGauge(getMetricsName(HoodieTimeline.CLEAN_ACTION,
DURATION_STR), durationInMs);
+ metrics.registerGauge(getMetricsName(HoodieTimeline.CLEAN_ACTION,
DELETE_FILES_NUM_STR), numFilesDeleted);
+}
+ }
+
+ public void updateArchiveMetrics(long durationInMs, int numFilesDeleted) {
+if (config.isMetricsOn()) {
+ LOG.info(
+ String.format("Sending archive metrics (duration=%d,
numFilesDeleted=%d)", durationInMs, numFilesDeleted));
+ metrics.registerGauge(getMetricsName(ARCHIVE_ACTION, DURATION_STR),
durationInMs);
+ metrics.registerGauge(getMetricsName(ARCHIVE_ACTION,
DELETE_FILES_NUM_STR), numFilesDeleted);
}
}
public void updateFinalizeWriteMetrics(long durationInMs, long
numFilesFinalized) {
if (config.isMetricsOn()) {
LOG.info(String.format("Sending finalize write metrics (duration=%d,
numFilesFinalized=%d)", durationInMs,
numFilesFinalized));
- metrics.registerGauge(getMetricsName("finalize", "duration"),
durationInMs);
- metrics.registerGauge(getMetricsName("finalize", "numFilesFinalized"),
numFilesFinalized);
+ metrics.registerGauge(getMetricsName(FINALIZE_ACTION, DURATION_STR),
durationInMs);
+ metrics.registerGauge(getMetricsName(FINALIZE_ACTION,
FINALIZED_FILES_NUM_STR), numFilesFinalized);
}
}
public void updateIndexMetrics(final String action, final long durationInMs)
{
if (config.isMetricsOn()) {
LOG.info(String.format("Sending index metrics (%s.duration, %d)",
action, durationInMs));
Review Comment:
We can also update the string literal in the log here
##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/HoodieMetrics.java:
##
@@ -92,20 +106,21 @@ public HoodieMetrics(HoodieWriteConfig config) {
this.tableName = config.getTableName();
if (config.isMetricsOn()) {
metrics = Metrics.getInstance(config);
- this.rollbackTimerName = getMetricsName("timer",
HoodieTimeline.ROLLBACK_ACTION);
- this.cleanTimerName = getMetricsName("timer",
HoodieTimeline.CLEAN_ACTION);
- this.commitTimerName = getMetricsName("timer",
HoodieTimeline.COMMIT_ACTION);
- this.deltaCommitTimerName = getMetricsName("timer",
HoodieTimeline.DELTA_COMMIT_ACTION);
- this.replaceCommitTimerName = getMetricsName("tim
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1812240602 ## CI report: * 2f97634b8b59e9f61dc05b649e78f9fe747c5ee5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20922) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1812151028 ## CI report: * 40eab84a79dace5e163dfd66958bb4993942b0a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20919) * 2f97634b8b59e9f61dc05b649e78f9fe747c5ee5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20922) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
majian1998 closed pull request #10101: [HUDI-7099] Providing metrics for archive and defining some string constants URL: https://github.com/apache/hudi/pull/10101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7099] Providing metrics for archive and defining some string constants [hudi]
hudi-bot commented on PR #10101: URL: https://github.com/apache/hudi/pull/10101#issuecomment-1812138721 ## CI report: * 40eab84a79dace5e163dfd66958bb4993942b0a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20919) * 2f97634b8b59e9f61dc05b649e78f9fe747c5ee5 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
