[
https://issues.apache.org/jira/browse/NIFI-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984680#comment-15984680
]
Joseph Witt commented on NIFI-71:
---------------------------------
[~markap14] I believe this ask from Andre is for you.
[~trixpan] "necrobumping" I definitely learned a new word today.
> Persistent Prov Repo should compress in blocks
> ----------------------------------------------
>
> Key: NIFI-71
> URL: https://issues.apache.org/jira/browse/NIFI-71
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Matt Gilman
> Priority: Minor
>
> Currently we write a bunch of events to a file and then compress the file. We
> then index the file offset of the uncompressed version of the file.
> We should instead compress in chunks of X number of events of X number of
> bytes. Then index the offset of the chunk in the compressed version. This
> way, we can use FileInputStream.skip to seek to the appropriate offset and
> then wrap the stream in GZIPInputStream. This allwos us to avoid reading a
> lot of compressed data to get to the desired offset.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
