On 02/19/2018 12:08 PM, Jan Pokorný wrote:
> On 09/02/18 17:55 -0600, Ken Gaillot wrote:
>> On Fri, 2018-02-09 at 18:54 -0500, Digimer wrote:
>>> On 2018-02-09 06:51 PM, Ken Gaillot wrote:
On Fri, 2018-02-09 at 12:52 -0500, Digimer wrote:
> On 2018-02-09 03:27 AM, Jan Pokorný wrote:
>> there is certainly whole can of these worms, put first that
>> crosses my mind: performing double (de)compression on two levels
>> of abstraction in the inter-node communication is not very
>> clever, to put it mildly.
>>
>> So far, just pacemaker was doing that for itself under certain
>> conditions, now corosync 3 will have it's iron in this fire
>> through kronosnet, too. Perhaps something to keep in mind to
>> avoid exercises in futility.
> Can pacemaker be told to not do compression? If not, can that be
> added in pacemaker v2?
Or better yet, is there some corosync API call we can use to
determine whether corosync/knet is using compression?
There's currently no way to turn compression off in Pacemaker,
however it is only used for IPC messages that pass a fairly high
size threshold, so many clusters would be unaffected even without
changes.
>>> Can you "turn off compression" but just changing that threshold to
>>> some silly high number?
>> It's hardcoded, so you'd have to edit the source and recompile.
> FTR, since half year ago, I've had some resources noted for further
> investigation on this topic of pacemaker-level compression -- since
> it compresses XML, there are some specifics of the input that sugggest
> more effective processing is possible.
>
> Indeed; there's a huge, rigorously maintained non-binary files
> compression benchmark that coincidentally also aims at XML files
> (despite presumably more text-oriented than structure-oriented):
>
> http://mattmahoney.net/dc/text.html
>
> Basically, I can see two (three) categories of possible optimizations:
>
> 0. pre-fill the scan dictionary for the compression algorithm
>with sequences that are statistically (constantly) most frequent
>(a priori known tag names?)
>
> 1. preprocessing of XML to allow for more efficient generic
>compression (like with bzip2 that is currently utilized), e.g.
>
>* XMill
> - https://homes.cs.washington.edu/~suciu/XMILL/
>
>* XWRT (XML-WRT)
> - https://github.com/inikep/XWRT
>
> 2. more effiecient algorithms as such for non-binary payloads
>(the benchmark above can help with selection of the candidates)
>
> * * *
>
> That being said, there are legitimate reasons to want merely the
> high-level messaging be involved with compression, because that's
> the only layer intimate with the respective application-specific
> data and hence can provide optimal compression methods beyond
> the reach of the generic ones.
Alternatively high-level might pass some hint for compression
with the data/config (like e.g. binary, xml, c-source, ...) for the
lower layer to be able to efficiently choose the algorithm.
e.g. different/optimized compression per cpg and again something
different for virtual bridging e.g. would be possible.
Regards,
Klaus
>
>
>
> ___
> Developers mailing list
> Developers@clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/developers
___
Developers mailing list
Developers@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/developers