[
https://issues.apache.org/jira/browse/IGNITE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958546#comment-15958546
]
Vladimir Ozerov commented on IGNITE-3592:
-----------------------------------------
[~daradurvs],
I reviewed the code. Looks like current implementation is not complete, or may
be I do not fully understood the goals. All in all, I propose to move the
ticket to the next version (2.1), so that we can implement it in a clean and
consistent fashion without any rush, as community expected Apache Ignite 2.0 to
be delivered within several weeks.
My concerns:
1) I am not quite understand what and when gets compressed/decompressed. For
example, even if binary object was compressed during serialization, on primary
node it will be stored in memory in uncompressed form (due to eager
decompression in {{BinarObject}} constructor. Looks like there is no
improvement in memory usage.
2) Compression is applied not only to user provided objects, but to Ignite
infrastructure as well. You can observe it by setting a breakpoint to
{{compress}} method and launching any test. Lot's of Ignite internals get
compressed and decompressed over and over again, what leads to additional
overhead while saving only very few bits of data on network layer.
3) I do not think that pluggable compression is required immediately. Instead,
we can provide 1-2 well-known base algorithms (such as GZIP) for now. Otherwise
it would be difficult to maintain interoperability with other platforms.
4) There is no integration with .NET and CPP layers which both rely on binary
format.
5) I would avoid per-field serialization for now. It increases complexity while
adding little value to end users.
Having said that I would do a step back and define our goals first. What are we
going to compress? If we want to compress network messages, then it should be
implemented on {{TcpDiscoverySpi}} layer. I we want to compress cache objects,
it should be done on per-cache layer (not only globally) making sure that data
is really stored in compressed form, and possibly this should be done not
inside binary objects, but somewhere around cache storage logic (e.g.
{{GridCacheMapEntry}}). Another important thing to keep in mind is ticket
IGNITE-3477 which is to be merged soon. It introduces a lot of changes to
storage logic, so probably we should merge it first before starting working on
compression.
Having said that, I propose to stabilize design and public API first.
[~kuaw26], could you please clarify what case did you have in mind when
creating this ticket?
> Provide some kind of pluggable compression SPI support
> ------------------------------------------------------
>
> Key: IGNITE-3592
> URL: https://issues.apache.org/jira/browse/IGNITE-3592
> Project: Ignite
> Issue Type: Task
> Components: cache
> Reporter: Alexey Kuznetsov
> Assignee: Vyacheslav Daradur
> Fix For: 2.0
>
>
> It may be useful in some cases to compress data stored in cache.
> And in order to give access to compressed data from SQL engine this support
> should be implemented in ignite-core level.
> See discussion on dev-list:
> http://apache-ignite-developers.2346864.n4.nabble.com/Data-compression-in-Ignite-2-0-td10099.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
