This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch rfc-15 in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 8bdbdf4 [HUDI-842] Reader and Writer for Metadata. new 389be1b [RFC-15] Add metrics to track the time for each file system call. new e8f1698 [RFC-15] Added a distributed metrics registry for spark which can be used to collect metrics from executors. new ce1c983 [RFC-15] Using distributed metrics registry in metadata code. new 7bea565 [MINOR] Reduced number of fs calls to check for table existence. Removed dead code. new a2dcacf [HUDI-1346] Choose a new instant time when performing autoClean. new b16586d [HUDI-1321] Created HoodieMetadataConfig to specify configuration for the metadata table. new 6ade850 [HUDI-1317] Fix initialization when Async jobs are scheduled. new 48d2f80 [RFC-15] Fixing checkstyle errors. new 86fdd65 [HUDI-1305] Added an API to shutdown and remove the metrics reporter. (#2132) new 9e8025d [RFC-15] Fixing metrics printing to console which generates a lot of text. new 5e13b19 [RFC-15] Fixing checkstyle errors. new 34872bb [RFC-15] Reverting change to AbstractHoodieLogRecordScanner as they lead to hung tests. new 4b7a4b3 [RFC-15] Fixing getFileSystemView() which requires getCompletedCommitsTimeline(). new d305948 [RFC-15] Fixing code review comments The 14 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/hudi/cli/commands/MetadataCommand.java | 23 +- .../apache/hudi/client/AbstractHoodieClient.java | 27 +- .../hudi/client/AbstractHoodieWriteClient.java | 5 + .../apache/hudi/client/AsyncCleanerService.java | 7 +- .../org/apache/hudi/client/HoodieWriteClient.java | 48 ++-- .../apache/hudi/config/HoodieMetadataConfig.java | 151 +++++++++++ .../apache/hudi/config/HoodieMetricsConfig.java | 8 + .../org/apache/hudi/config/HoodieWriteConfig.java | 85 +++---- .../apache/hudi/metadata/HoodieMetadataWriter.java | 122 ++++++--- .../hudi/metrics/ConsoleMetricsReporter.java | 3 + .../apache/hudi/metrics/DistributedRegistry.java | 107 ++++++++ .../main/java/org/apache/hudi/metrics/Metrics.java | 30 ++- .../java/org/apache/hudi/table/HoodieTable.java | 9 +- .../apache/hudi/metadata/TestHoodieMetadata.java | 99 ++++--- .../hudi/metrics/TestHoodieConsoleMetrics.java | 6 + .../java/org/apache/hudi/common/fs/FSUtils.java | 17 -- .../hudi/common/fs/HoodieWrapperFileSystem.java | 283 +++++++++++++-------- .../common/fs/SizeAwareFSDataOutputStream.java | 17 +- .../hudi/common/fs/TimedFSDataInputStream.java | 79 ++++++ .../metrics/{Registry.java => LocalRegistry.java} | 62 +---- .../org/apache/hudi/common/metrics/Registry.java | 108 ++++---- .../table/log/AbstractHoodieLogRecordScanner.java | 14 +- .../hudi/common/table/log/HoodieLogFileReader.java | 5 +- .../hudi/exception/TableNotFoundException.java | 14 +- .../apache/hudi/metadata/HoodieMetadataReader.java | 105 ++++---- 25 files changed, 971 insertions(+), 463 deletions(-) create mode 100644 hudi-client/src/main/java/org/apache/hudi/config/HoodieMetadataConfig.java create mode 100644 hudi-client/src/main/java/org/apache/hudi/metrics/DistributedRegistry.java create mode 100644 hudi-common/src/main/java/org/apache/hudi/common/fs/TimedFSDataInputStream.java copy hudi-common/src/main/java/org/apache/hudi/common/metrics/{Registry.java => LocalRegistry.java} (58%)