Github user iamaleksey commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/239#discussion_r204874237
--- Diff:
src/java/org/apache/cassandra/db/streaming/CassandraOutgoingFile.java ---
@@ -114,13 +155,51 @@ public void write(StreamSession session,
DataOutputStreamPlus out, int version)
CassandraStreamHeader.serializer.serialize(header, out, version);
out.flush();
- CassandraStreamWriter writer = header.compressionInfo == null ?
- new CassandraStreamWriter(sstable,
header.sections, session) :
- new
CompressedCassandraStreamWriter(sstable, header.sections,
-
header.compressionInfo, session);
+ IStreamWriter writer;
+ if (shouldStreamFullSSTable())
+ {
+ writer = new CassandraBlockStreamWriter(sstable, session,
components);
+ }
+ else
+ {
+ writer = (header.compressionInfo == null) ?
+ new CassandraStreamWriter(sstable, header.sections,
session) :
+ new CompressedCassandraStreamWriter(sstable,
header.sections,
+
header.compressionInfo, session);
+ }
writer.write(out);
}
+ @VisibleForTesting
+ public boolean shouldStreamFullSSTable()
+ {
+ return isFullSSTableTransfersEnabled && isFullyContained;
+ }
+
+ @VisibleForTesting
+ public boolean fullyContainedIn(List<Range<Token>> normalizedRanges,
SSTableReader sstable)
+ {
+ if (normalizedRanges == null)
+ return false;
+
+ RangeOwnHelper rangeOwnHelper = new
RangeOwnHelper(normalizedRanges);
+ try (KeyIterator iter = new KeyIterator(sstable.descriptor,
sstable.metadata()))
+ {
+ while (iter.hasNext())
+ {
+ DecoratedKey key = iter.next();
+ try
+ {
+ rangeOwnHelper.check(key);
+ } catch(RuntimeException e)
--- End diff --
@dineshjoshi Duplicating a reply here: I think it's fine to not block this
PR on the improvements, but disable zero copy for STCS for now, and commit that
way. Once we optimise the containment check - ideally before 4.0 - enable for
STCS as well.
Encoding information about the effective contained ranges is one way.
Alternative way would be to invert the ranges we are checking for, and look for
presence of keys in the gaps - but relying on summary and index to efficiently
search rather than linearly scan. But again - let's temporarily disable zero
copy for STCS for now, to not delay the ticket beyond necessary.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
