keith-turner commented on code in PR #4750:
URL: https://github.com/apache/accumulo/pull/4750#discussion_r1690245443
##########
server/manager/src/main/java/org/apache/accumulo/manager/Manager.java:
##########
@@ -712,21 +708,35 @@ public void run() {
* migration will refer to a non-existing tablet, so it can never
complete. Periodically scan
* the metadata table and remove any migrating tablets that no longer
exist.
*/
- private void cleanupNonexistentMigrations(final AccumuloClient
accumuloClient)
+ private void cleanupNonexistentMigrations(final ClientContext
clientContext)
throws TableNotFoundException {
- Scanner scanner = accumuloClient.createScanner(MetadataTable.NAME,
Authorizations.EMPTY);
- TabletColumnFamily.PREV_ROW_COLUMN.fetch(scanner);
- scanner.setRange(MetadataSchema.TabletsSection.getRange());
- Set<KeyExtent> notSeen;
+
+ Map<DataLevel,Set<KeyExtent>> notSeen = new EnumMap<>(DataLevel.class);
+
synchronized (migrations) {
- notSeen = new HashSet<>(migrations.keySet());
+ // partition migrations by level
+ migrations.keySet().forEach(extent -> notSeen
+ .computeIfAbsent(DataLevel.of(extent.tableId()), k -> new
HashSet<>()).add(extent));
}
- for (Entry<Key,Value> entry : scanner) {
- KeyExtent extent = KeyExtent.fromMetaPrevRow(entry);
- notSeen.remove(extent);
+
+ // for each level find the set of migrating tablets that do not exists
in metadata store
+ for (DataLevel dataLevel : DataLevel.values()) {
+ var notSeenForLevel = notSeen.getOrDefault(dataLevel, Set.of());
+ if (notSeenForLevel.isEmpty() || dataLevel == DataLevel.ROOT) {
+ // No need to scan this level if there are no migrations. The root
tablet is always
+ // expected to exists, so no need to read its metadata.
+ continue;
+ }
+
+ try (var tablets =
clientContext.getAmple().readTablets().forLevel(dataLevel)
+ .fetch(TabletMetadata.ColumnType.PREV_ROW).build()) {
+ tablets.forEach(tabletMeta ->
notSeenForLevel.remove(tabletMeta.getExtent()));
Review Comment:
I added a comment about an an undocumented goal in the code that would not
be obvious from reading the code, especially with an abstraction layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
