[
https://issues.apache.org/jira/browse/ACCUMULO-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211402#comment-14211402
]
Eric Newton commented on ACCUMULO-3282:
---------------------------------------
I was hoping for something more comprehensive: a class that would model the
data in the metadata table. Still, I'm sure this will accelerate initial
tablet assignment.
> Reduce scans of metadata when assigning tablet
> ----------------------------------------------
>
> Key: ACCUMULO-3282
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3282
> Project: Accumulo
> Issue Type: Improvement
> Components: tserver
> Reporter: Josh Elser
> Assignee: Josh Elser
> Fix For: 1.7.0
>
> Attachments: ACCUMULO-3282.patch
>
>
> Noticed the following case digging through the assignment code:
> {code:title=TabletServer.java}
> public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent
> extent, TServerInstance instance, SortedMap<Key,Value> tabletsKeyValues,
> String clientAddress, ZooLock lock) throws AccumuloSecurityException,
> DistributedStoreException, AccumuloException {
> ...
> public static Pair<Text,KeyExtent> verifyTabletInformation(KeyExtent
> extent, TServerInstance instance, SortedMap<Key,Value> tabletsKeyValues,
> String clientAddress, ZooLock lock) throws AccumuloSecurityException,
> DistributedStoreException, AccumuloException {
> log.debug("verifying extent " + extent);
> if (extent.isRootTablet()) {
> return verifyRootTablet(extent, instance);
> }
> String tableToVerify = MetadataTable.ID;
> if (extent.isMeta())
> tableToVerify = RootTable.ID;
> List<ColumnFQ> columnsToFetch = Arrays.asList(new ColumnFQ[]
> {TabletsSection.ServerColumnFamily.DIRECTORY_COLUMN,
> TabletsSection.TabletColumnFamily.PREV_ROW_COLUMN,
> TabletsSection.TabletColumnFamily.SPLIT_RATIO_COLUMN,
> TabletsSection.TabletColumnFamily.OLD_PREV_ROW_COLUMN,
> TabletsSection.ServerColumnFamily.TIME_COLUMN});
> ScannerImpl scanner = new ScannerImpl(HdfsZooInstance.getInstance(),
> SystemCredentials.get(), tableToVerify, Authorizations.EMPTY);
> scanner.setRange(extent.toMetadataRange());
> TreeMap<Key,Value> tkv = new TreeMap<Key,Value>();
> for (Entry<Key,Value> entry : scanner)
> tkv.put(entry.getKey(), entry.getValue());
> // only populate map after success
> if (tabletsKeyValues == null) {
> tabletsKeyValues = tkv;
> } else {
> tabletsKeyValues.clear();
> tabletsKeyValues.putAll(tkv);
> {code}
> Essentially, we read a few columns for the tablet's row from metadata and
> (when metadata is in a consistent state) we return those columns to the
> Tablet constructor.
> {code:title=Tablet.java}
> private Tablet(TabletServer tabletServer, Text location, KeyExtent extent,
> TabletResourceManager trm, Configuration conf, VolumeManager fs,
> SortedMap<Key,Value> tabletsKeyValues) throws IOException {
> this(tabletServer, location, extent, trm, conf, fs,
> lookupLogEntries(extent, tabletsKeyValues),
> lookupDatafiles(tabletServer.getSystemConfiguration(), fs,
> extent, tabletsKeyValues),
> lookupTime(tabletServer.getSystemConfiguration(), extent, tabletsKeyValues),
> lookupLastServer(extent, tabletsKeyValues),
> lookupScanFiles(extent, tabletsKeyValues, fs), lookupFlushID(extent,
> tabletsKeyValues), lookupCompactID(extent, tabletsKeyValues));
> }
> {code}
> {{lookupDataFiles}} reaches back out to the metadata table to fetch this
> column. I'm not sure if there's a reason why we can't do this all at once.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
