[
https://issues.apache.org/jira/browse/ACCUMULO-4419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433018#comment-15433018
]
Josh Elser commented on ACCUMULO-4419:
--------------------------------------
Thanks, Marc! This is exactly what I was looking for. Some further thoughts:
bq. Internally we've used the non pooled method since CodecPool did not trim
the compressors when they weren't being used. In some cases a large number of
sources will increase the memory foot print, so this was for memory usage;
however, the ability to choose a factory was something I added to accommodate
those who wanted to make their own choice.
So, avoid the pool entirely to make sure when a "scan storm" happens, you don't
hold on to compressors for any longer than absolutely necessary?
bq. In testing I found significant performance increases by changing the 1K
buffer higher.
Neat, is this reproducible at all? Seems like some sort of harness would be
useful in ever revisiting this kind of testing.
bq. I didn't want to confuse the focus of the PR, but I did want to allow the
CompressorFactory implementation and its configuration the ability to be
updated.
Understood and appreciated. Dynamic configuration is super helpful and (IMO)
the norm now.
> Create Compressor factory allowing Compression settings to be updated
> ---------------------------------------------------------------------
>
> Key: ACCUMULO-4419
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4419
> Project: Accumulo
> Issue Type: Improvement
> Reporter: marco polo
> Assignee: marco polo
> Priority: Minor
> Labels: core
> Fix For: 1.7.3, 1.8.1
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> This ticket is to account for work done elsewhere in which I've made the
> compression pool configurable such that we either don't use the pool at all
> or use an adjustable pool based on commons-pool
> Other configuration options are now updated through a CompressionUpdate
> mechanism.
> This PR will move us away from CodecPool, but will allow us greater control
> over trimming codecs from the pool itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
