[
https://issues.apache.org/jira/browse/PHOENIX-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837216#comment-15837216
]
Thomas D'Silva commented on PHOENIX-3585:
-----------------------------------------
[~rajeshbabu]
The test was failing TransactionProcessor.preCompactScannerOpen runs first
(because it has a higher priority) and returns an internal scanner which gets
passed into the IndexHalfStoreFileReaderGenerator.preCompactScannerOpen() call
as the InternalScanner so the LocalIndexStoreFileScanner is never created.
{code}
@Override
public InternalScanner
preCompactScannerOpen(ObserverContext<RegionCoprocessorEnvironment> c,
Store store, List<? extends KeyValueScanner> scanners, ScanType
scanType,
long earliestPutTs, InternalScanner s, CompactionRequest request)
throws IOException {
if (!store.getFamily().getNameAsString()
.startsWith(QueryConstants.LOCAL_INDEX_COLUMN_FAMILY_PREFIX)
|| s != null // -----> returns here
|| !store.hasReferences()) {
return s;
}
List<StoreFileScanner> newScanners = new
ArrayList<StoreFileScanner>(scanners.size());
Scan scan = new Scan();
scan.setMaxVersions(store.getFamily().getMaxVersions());
boolean scanUsePread =
c.getEnvironment().getConfiguration().getBoolean("hbase.storescanner.use.pread",
scan.isSmall());
for(KeyValueScanner scanner: scanners) {
Reader reader = ((StoreFileScanner) scanner).getReaderForTesting();
if (reader instanceof IndexHalfStoreFileReader) {
newScanners.add(new LocalIndexStoreFileScanner(reader,
reader.getScanner(
scan.getCacheBlocks(), scanUsePread, false), true,
reader.getHFileReader()
.hasMVCCInfo(), store.getSmallestReadPoint()));
} else {
newScanners.add(((StoreFileScanner) scanner));
}
}
return new StoreScanner(store, store.getScanInfo(), scan, newScanners,
scanType, store.getSmallestReadPoint(), earliestPutTs);
}
{code}
[~jamestaylor]
The TransactionProcessor creates a scan with a filter to filter out invalid
data. Do you see any way we can combine the StoreScanner from the
TransactionProcessor with that created in IndexHalfStoreFileReaderGenerator
{code}
protected InternalScanner createStoreScanner(RegionCoprocessorEnvironment env,
String action,
TransactionVisibilityState
snapshot, Store store,
List<? extends KeyValueScanner>
scanners, ScanType type,
long earliestPutTs) throws
IOException {
if (snapshot == null) {
if (LOG.isDebugEnabled()) {
LOG.debug("Region " + env.getRegion().getRegionNameAsString() +
", no current transaction state found, defaulting to normal
" + action + " scanner");
}
return null;
}
// construct a dummy transaction from the latest snapshot
Transaction dummyTx = TxUtils.createDummyTransaction(snapshot);
Scan scan = new Scan();
// need to see all versions, since we filter out excludes and applications
may rely on multiple versions
scan.setMaxVersions();
scan.setFilter(
new IncludeInProgressFilter(dummyTx.getVisibilityUpperBound(),
snapshot.getInvalid(),
getTransactionFilter(dummyTx, type, null)));
return new StoreScanner(store, store.getScanInfo(), scan, scanners,
type, store.getSmallestReadPoint(), earliestPutTs);
}
{code}
> MutableIndexIT testSplitDuringIndexScan and testIndexHalfStoreFileReader fail
> for transactional tables and local indexes
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3585
> URL: https://issues.apache.org/jira/browse/PHOENIX-3585
> Project: Phoenix
> Issue Type: Bug
> Reporter: Thomas D'Silva
> Attachments: diff.patch
>
>
> the tests fail if we use HDFSTransactionStateStorage instead of
> InMemoryTransactionStateStorage when we create the TransactionManager in
> BaseTest
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
