Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

yihua merged PR #10641:
URL: https://github.com/apache/hudi/pull/10641


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

lokeshj1703 commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2136528074

   @yihua can we merge this PR? The CI is green now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2136077033

   
   ## CI report:
   
   * b86a5261475a9a24449bab2be28ced124587e242 Azure: 
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24014)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2121936991

   
   ## CI report:
   
   * b086907bcddcb493e9d7f9a711b74e0fc7d1ea48 Azure: 
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23877)
 
   * b86a5261475a9a24449bab2be28ced124587e242 UNKNOWN
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2108796212

   
   ## CI report:
   
   * b086907bcddcb493e9d7f9a711b74e0fc7d1ea48 Azure: 
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23877)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2108687096

   
   ## CI report:
   
   * fc072259ead8a9870a1b26b5aceb7882aabebb32 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22414)
 
   * b086907bcddcb493e9d7f9a711b74e0fc7d1ea48 Azure: 
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23877)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2108674779

   
   ## CI report:
   
   * fc072259ead8a9870a1b26b5aceb7882aabebb32 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22414)
 
   * b086907bcddcb493e9d7f9a711b74e0fc7d1ea48 UNKNOWN
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

prashantwason commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2045987807

   Sure. I can put out a patch for b. later.
   If you have tested the writer side metrics emitted as per this change for
   MDT then this is good to go.
   
   On Mon, Apr 8, 2024 at 8:50 PM Sivabalan Narayanan ***@***.***>
   wrote:
   
   > hey @prashantwason  : lets de-couple
   > the fixes.
   > a. Fixing MDT to emit writer side metrics(commit duration, compaction
   > duration etc)
   > b. Fixing MDT to emit reader side metrics (col stats look up duration etc)
   > during distributed registry.
   >
   > I feel we should focus on (a) in this patch and get it landed. and you can
   > put out a patch (I assume you folks already have a fix) for distributed
   > registry based metrics from the executor.
   >
   > If you are aligned on that, let us know if you have any feedback on this
   > patch. or if we are good to go ahead.
   >
   > —
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

nsivabalan commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2044100016

   hey @prashantwason : lets de-couple the fixes. 
   a. Fixing MDT to emit writer side metrics(commit duration, compaction 
duration etc)
   b. Fixing MDT to emit reader side metrics (col stats look up duration etc) 
during distributed registry. 
   
   I feel we should focus on (a) in this patch and get it landed. and you can 
put out a patch (I assume you folks already have a fix) for distributed 
registry based metrics from the executor. 
   
   If you are aligned on that, let us know if you have any feedback on this 
patch. or if we are good to go ahead. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

prashantwason commented on code in PR #10641:
URL: https://github.com/apache/hudi/pull/10641#discussion_r1556617105


##
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMetrics.java:
##
@@ -136,7 +144,7 @@ public void updateMetrics(String action, long durationInMs) 
{
 String countKey = action + ".count";
 String durationKey = action + ".totalDuration";
 incrementMetric(countKey, 1);
-incrementMetric(durationKey, durationInMs);
+setMetric(durationKey, durationInMs);

Review Comment:
   You are assuming that code calling these functions would only call once. 
That may not be a correct assumption for all cases - opening the MDT is costly 
so multiple lookups etc can be called on open MDT readers. 



##
hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java:
##
@@ -302,8 +303,8 @@ public Map 
readRecordIndex(List reco
 });
 
 metrics.ifPresent(m -> 
m.updateMetrics(HoodieMetadataMetrics.LOOKUP_RECORD_INDEX_TIME_STR, 
timer.endTimer()));
-metrics.ifPresent(m -> 
m.updateMetrics(HoodieMetadataMetrics.LOOKUP_RECORD_INDEX_KEYS_COUNT_STR, 
recordKeys.size()));
-metrics.ifPresent(m -> 
m.updateMetrics(HoodieMetadataMetrics.LOOKUP_RECORD_INDEX_KEYS_HITS_COUNT_STR, 
recordKeyToLocation.size()));
+metrics.ifPresent(m -> 
m.setMetric(HoodieMetadataMetrics.LOOKUP_RECORD_INDEX_KEYS_COUNT_STR, 
recordKeys.size()));

Review Comment:
   The same HoodieTableMetadata object can be used to lookup keys from MDT 
multiple times. In that case, update is more accurate.
   



##
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMetrics.java:
##
@@ -73,10 +79,12 @@ public class HoodieMetadataMetrics implements Serializable {
 
   private static final Logger LOG = 
LoggerFactory.getLogger(HoodieMetadataMetrics.class);
 
-  private final Registry metricsRegistry;
+  private final transient MetricRegistry metricsRegistry;
+  private final transient Metrics metrics;
 
-  public HoodieMetadataMetrics(Registry metricsRegistry) {
-this.metricsRegistry = metricsRegistry;
+  public HoodieMetadataMetrics(HoodieMetricsConfig metricsConfig) {

Review Comment:
   If you do not use Registry then no metrics can be collected from the 
executors where most of the operations on the MDT readers take place (for 
indexes other than files index).
   
   Eg. RI lookup -> since there are multiple file groups in record_index, when 
looking up keys from the record index, each executor opens one file group of 
the record index and reads the keys that belong to that file group. When 
HoodieTableMetadata is serialized by Spark and send to the executors, the 
executors end up updating a local copy of the metadata metrics. since the 
publishing of the metrics is only done on the driver side, the metrics updated 
on the executor side never make it to the driver side and hence never published.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

nsivabalan commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2041490421

   reviewed last 2 commits. LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

nsivabalan commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-2041483567

   I guess this is stacked ontop of 10635. Can you add a link to PR description 
to the actual diff to review for this patch(ignoring the stacked PR changed). 
   If I am not wrong, 
https://github.com/apache/hudi/pull/10641/files/adc183a351b8f15d671c0c6eefd1f999bed54774..fc072259ead8a9870a1b26b5aceb7882aabebb32
 
   is the right link (last 2 commits). 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1938517524

   
   ## CI report:
   
   * fc072259ead8a9870a1b26b5aceb7882aabebb32 Azure: 
[FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22414)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1938245607

   
   ## CI report:
   
   * 30b747a62bf9d4a9503bd992764f6f4f7cbe3601 Azure: 
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22374)
 
   * fc072259ead8a9870a1b26b5aceb7882aabebb32 Azure: 
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22414)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1938236429

   
   ## CI report:
   
   * 30b747a62bf9d4a9503bd992764f6f4f7cbe3601 Azure: 
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22374)
 
   * fc072259ead8a9870a1b26b5aceb7882aabebb32 UNKNOWN
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

lokeshj1703 commented on code in PR #10641:
URL: https://github.com/apache/hudi/pull/10641#discussion_r1485825887


##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##
@@ -147,6 +148,10 @@ public static HoodieWriteConfig createMetadataWriteConfig(
 final Properties properties = new Properties();
 properties.put(HoodieTableConfig.RECORDKEY_FIELDS.key(), 
RECORD_KEY_FIELD_NAME);
 properties.put("hoodie.datasource.write.recordkey.field", 
RECORD_KEY_FIELD_NAME);
+if (nonEmpty(writeConfig.getMetricReporterMetricsNamePrefix())) {
+  properties.put(HoodieMetricsConfig.METRICS_REPORTER_PREFIX.key(),
+  writeConfig.getMetricReporterMetricsNamePrefix() + "_metadata");
+}

Review Comment:
   Yes you are right. I have made a change to use `METADATA_TABLE_NAME_SUFFIX` 
instead here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

lokeshj1703 commented on code in PR #10641:
URL: https://github.com/apache/hudi/pull/10641#discussion_r1485824879


##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -324,7 +324,7 @@ private boolean metadataTableExists(HoodieTableMetaClient 
dataMetaClient) throws
 }
 
 if (reInitialize) {
-  metrics.ifPresent(m -> 
m.updateMetrics(HoodieMetadataMetrics.REBOOTSTRAP_STR, 1));
+  metrics.ifPresent(m -> 
m.incrementMetric(HoodieMetadataMetrics.REBOOTSTRAP_STR, 1));
   LOG.info("Deleting Metadata Table directory so that it can be 
re-initialized");

Review Comment:
   This metric is for rebootstrapping. `updateMetrics` is updating two metrics 
- number of times rebootstrap is called and amount of time the action took. 
There is no good way to calculate the time so I simplified the metric to track 
only the number of times rebootstrap is called.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

nsivabalan commented on code in PR #10641:
URL: https://github.com/apache/hudi/pull/10641#discussion_r1485785454


##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##
@@ -147,6 +148,10 @@ public static HoodieWriteConfig createMetadataWriteConfig(
 final Properties properties = new Properties();
 properties.put(HoodieTableConfig.RECORDKEY_FIELDS.key(), 
RECORD_KEY_FIELD_NAME);
 properties.put("hoodie.datasource.write.recordkey.field", 
RECORD_KEY_FIELD_NAME);
+if (nonEmpty(writeConfig.getMetricReporterMetricsNamePrefix())) {
+  properties.put(HoodieMetricsConfig.METRICS_REPORTER_PREFIX.key(),
+  writeConfig.getMetricReporterMetricsNamePrefix() + "_metadata");
+}

Review Comment:
   guess, we are deriving the metrics name prefix from data table's write 
config. and hence suffixing "_metadata" to it. 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

danny0405 commented on code in PR #10641:
URL: https://github.com/apache/hudi/pull/10641#discussion_r1485703297


##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##
@@ -324,7 +324,7 @@ private boolean metadataTableExists(HoodieTableMetaClient 
dataMetaClient) throws
 }
 
 if (reInitialize) {
-  metrics.ifPresent(m -> 
m.updateMetrics(HoodieMetadataMetrics.REBOOTSTRAP_STR, 1));
+  metrics.ifPresent(m -> 
m.incrementMetric(HoodieMetadataMetrics.REBOOTSTRAP_STR, 1));
   LOG.info("Deleting Metadata Table directory so that it can be 
re-initialized");

Review Comment:
   Are we reporting multiple times for bootstrapping?



##
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##
@@ -147,6 +148,10 @@ public static HoodieWriteConfig createMetadataWriteConfig(
 final Properties properties = new Properties();
 properties.put(HoodieTableConfig.RECORDKEY_FIELDS.key(), 
RECORD_KEY_FIELD_NAME);
 properties.put("hoodie.datasource.write.recordkey.field", 
RECORD_KEY_FIELD_NAME);
+if (nonEmpty(writeConfig.getMetricReporterMetricsNamePrefix())) {
+  properties.put(HoodieMetricsConfig.METRICS_REPORTER_PREFIX.key(),
+  writeConfig.getMetricReporterMetricsNamePrefix() + "_metadata");
+}

Review Comment:
   Where does the `_metadata` constant string come from?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1934356784

   
   ## CI report:
   
   * 30b747a62bf9d4a9503bd992764f6f4f7cbe3601 Azure: 
[SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22374)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

lokeshj1703 commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1934064669

   The PR is rebased on top of HUDI-7391


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1934019516

   
   ## CI report:
   
   * 30b747a62bf9d4a9503bd992764f6f4f7cbe3601 Azure: 
[PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22374)
 
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

hudi-bot commented on PR #10641:
URL: https://github.com/apache/hudi/pull/10641#issuecomment-1934007669

   
   ## CI report:
   
   * 30b747a62bf9d4a9503bd992764f6f4f7cbe3601 UNKNOWN
   
   
   Bot commands
 @hudi-bot supports the following commands:
   
- `@hudi-bot run azure` re-run the last Azure build
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

[via GitHub]

lokeshj1703 opened a new pull request, #10641:
URL: https://github.com/apache/hudi/pull/10641

   ### Change Logs
   
   For some of the metrics type like duration we are using incrementMetric 
instead of setMetric.
   Also some of the redundant metrics are removed. For example a count type 
metric has both count and duration metric getting pushed even though duration 
is not calculated.
   File lookup count metric is added for bloom filter and column stat
   
   ### Impact
   
   NA
   
   ### Risk level (write none, low medium or high below)
   
   low
   
   ### Documentation Update
   
   NA
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org