rustyrazorblade commented on code in PR #211:
URL:
https://github.com/apache/cassandra-analytics/pull/211#discussion_r3325476578
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/SortedSSTableWriter.java:
##########
@@ -297,6 +311,25 @@ public synchronized void close(BulkWriterContext
writerContext) throws IOExcepti
validateSSTables(writerContext);
}
+ private void rebuildFilterComponents(@NotNull BulkWriterContext
writerContext, @NotNull Path outputDirectory,
+ @NotNull DirectoryStream.Filter<Path>
filter) throws IOException
+ {
+ LocalDataLayer layer = buildLocalDataLayer(writerContext,
outputDirectory, null);
+ for (Path dataFile : getDataFileStream(filter))
+ {
+ try
+ {
+ FileSystemSSTable ssTable = new FileSystemSSTable(dataFile,
false, BufferingInputStreamStats::doNothingStats);
+ writerContext.bridge().rebuildBloomFilter(layer.partitioner(),
layer.cqlTable(), ssTable, outputDirectory);
+ LOGGER.error("Rebuilt bloom filter for sstable {}", dataFile);
+ }
+ catch (Exception e)
+ {
+ LOGGER.warn("Failed to rebuild bloom filter for sstable {}",
dataFile, e);
Review Comment:
Ok. Mind just leaving a comment so it's clear that we're deliberately not
uploading a bloom filter in case of a failure? I don't expect we'll hit this
path ever, I just want it to be clear of the intention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
