Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince merged PR #13107: URL: https://github.com/apache/pinot/pull/13107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1633513536 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/SegmentGeneratorConfig.java: ## @@ -600,6 +616,11 @@ public String

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1632638123 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/SegmentGeneratorConfig.java: ## @@ -600,6 +616,11 @@ public String

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1631741645 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/name/UploadedRealtimeSegmentNameGenerator.java: ## @@ -0,0 +1,93 @@ +/** + * Licensed to the

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1631688317 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,47 @@ protected

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1631496448 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,180 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1630674736 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,45 @@ protected

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1630674736 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,45 @@ protected

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1630664998 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,180 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1630365372 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,180 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1628157120 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/name/UploadedRealtimeSegmentNameGenerator.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1626373676 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,49 @@ protected void

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1625655826 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManagerTest.java: ## @@ -378,6 +379,115 @@ public

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1625655826 ## pinot-segment-local/src/test/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManagerTest.java: ## @@ -378,6 +379,115 @@ public

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1625400329 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,51 @@ protected

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1625104449 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,51 @@ protected void

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1625083242 ## pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/name/UploadedRealtimeSegmentNameGenerator.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624940078 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,199 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624940078 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,199 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624938440 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,51 @@ protected

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624938073 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,199 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624820901 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,199 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624238350 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,157 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1624238350 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,157 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1622835600 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,157 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1622832355 ## pinot-common/src/main/java/org/apache/pinot/common/utils/Benchmark.java: ## @@ -0,0 +1,89 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1621458240 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,157 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1621456711 ## pinot-common/src/main/java/org/apache/pinot/common/utils/Benchmark.java: ## @@ -0,0 +1,89 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1619935544 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,44 @@ protected

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1619934716 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,131 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1619430793 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,131 @@ +/** + * Licensed to the Apache Software Foundation

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1619257347 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java: ## @@ -158,6 +158,44 @@ protected void

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1619054555 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,131 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1618286142 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,131 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1618004659 ## pinot-common/src/main/java/org/apache/pinot/common/utils/SegmentUtils.java: ## @@ -44,6 +44,16 @@ public static Integer getRealtimeSegmentPartitionId(String

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1617728542 ## pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-common/src/main/java/org/apache/pinot/plugin/ingestion/batch/common/SegmentGenerationTaskRunner.java:

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

tibrewalpratik17 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1617017946 ## pinot-common/src/main/java/org/apache/pinot/common/utils/UploadedRealtimeSegmentName.java: ## @@ -0,0 +1,134 @@ +/** + * Licensed to the Apache Software

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1605420533 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1605390671 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1604457365 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1604457365 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1604457365 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1604457365 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1604127022 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1604008348 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1602319612 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1603148256 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1603148256 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1602319612 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1602098764 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1602067423 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

klsince commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1600770456 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1598868561 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

ankitsultana commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1598813431 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

Jackie-Jiang commented on PR #13107: URL: https://github.com/apache/pinot/pull/13107#issuecomment-2108248763 cc @snleee @swaminathanmanish to take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

satishd commented on code in PR #13107: URL: https://github.com/apache/pinot/pull/13107#discussion_r1597953408 ## pinot-common/src/main/java/org/apache/pinot/common/metadata/segment/SegmentPartitionMetadata.java: ## @@ -48,6 +53,21 @@ public SegmentPartitionMetadata(

Re: [PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

codecov-commenter commented on PR #13107: URL: https://github.com/apache/pinot/pull/13107#issuecomment-2102011974 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13107?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report

[PR] [Backfill] allow externally partitioned segment uploads for upsert tables [pinot]

rohityadav1993 opened a new pull request, #13107: URL: https://github.com/apache/pinot/pull/13107 `feature` `release-notes` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.