Github user merlimat commented on the issue:
https://github.com/apache/incubator-pulsar/issues/523
I am not saying we cant, just trying to scope all the implications :)
If you check the size after compression you're not sure whether a
particular message will go through or not.
Example:
I have a 6MB JSON document. Most probably it will compress to 600KB, but I
cannot tell 100%. If the JSON is weird enough it might be 5.5MB.
So if I send this 6MB message it might get rejected by the client.
Typically the safe-size buffer for compression is _bigger_ that the
uncompressed content, to account for incompressible content plus the compressor
overhead.
In my view, compressing is an optimization to safe network/IO/disk space,
but the max size should be enforced irrespectively of that. So if you have 6MB
or 10MB entries, you should be able to configure the system to handle that,
with 100% assurance that it will processed correctly, instead of relying on the
assumption that compression will solve it, which might be true in 99.99% of the
cases but fail at unexpected times.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
