[
https://issues.apache.org/jira/browse/IGNITE-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667277#comment-16667277
]
ASF GitHub Bot commented on IGNITE-10048:
-----------------------------------------
GitHub user glukos opened a pull request:
https://github.com/apache/ignite/pull/5197
IGNITE-10048 Bounded iteration in standalone WAL iterator with compac…
…tion enabled may skip records
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-10048
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/5197.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5197
----
commit 8185c620a57bb6869b2068eee2ba1a7fec0d7170
Author: Ivan Rakov <irakov@...>
Date: 2018-10-29T14:17:29Z
IGNITE-10048 Bounded iteration in standalone WAL iterator with compaction
enabled may skip records
----
> Bounded iteration in standalone WAL iterator with compaction enabled may skip
> records
> -------------------------------------------------------------------------------------
>
> Key: IGNITE-10048
> URL: https://issues.apache.org/jira/browse/IGNITE-10048
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Rakov
> Priority: Major
> Fix For: 2.8
>
>
> Bounded iteration with non-zero start/end offsets may skip some records in
> intermediate segments. Reproducer (wal compaction should be enabled):
> {noformat}
> /**
> *
> */
> public void testBoundedIterationOverSeveralSegments() throws Exception {
> walCompactionEnabled = true;
> IgniteEx ig = (IgniteEx)startGrid();
> String archiveWalDir = getArchiveWalDirPath(ig);
> ig.cluster().active(true);
> IgniteCache<Object, Object> cache = ig.getOrCreateCache(
> new CacheConfiguration<>().setName("c-n").setAffinity(new
> RendezvousAffinityFunction(false, 32)));
> IgniteCacheDatabaseSharedManager sharedMgr =
> ig.context().cache().context().database();
> IgniteWriteAheadLogManager walMgr =
> ig.context().cache().context().wal();
> WALPointer fromPtr = null;
> int recordsCnt = WAL_SEGMENT_SIZE / 8 /* record size */ * 5;
> for (int i = 0; i < recordsCnt; i++) {
> WALPointer ptr = walMgr.log(new PartitionDestroyRecord(i, i));
> if (i == 100)
> fromPtr = ptr;
> }
> assertNotNull(fromPtr);
> cache.put(1, 1);
> forceCheckpoint();
> // Generate WAL segments for filling WAL archive folder.
> for (int i = 0; i < 2 *
> ig.configuration().getDataStorageConfiguration().getWalSegments(); i++) {
> sharedMgr.checkpointReadLock();
> try {
> walMgr.log(new SnapshotRecord(i, false),
> RolloverType.NEXT_SEGMENT);
> }
> finally {
> sharedMgr.checkpointReadUnlock();
> }
> }
> cache.put(2, 2);
> forceCheckpoint();
> U.sleep(5000);
> stopGrid();
> WALIterator it = new IgniteWalIteratorFactory(log)
> .iterator(new
> IteratorParametersBuilder().from((FileWALPointer)fromPtr).filesOrDirs(archiveWalDir));
> TreeSet<Integer> foundCounters = new TreeSet<>();
> it.forEach(x -> {
> WALRecord rec = x.get2();
> if (rec instanceof PartitionDestroyRecord)
> foundCounters.add(((WalRecordCacheGroupAware)rec).groupId());
> });
> assertEquals(new Integer(100), foundCounters.first());
> assertEquals(new Integer(recordsCnt - 1), foundCounters.last());
> assertEquals(recordsCnt - 100, foundCounters.size());
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
