Hello All,
First I would like to thank you for the prompt replies we received and
apologise for the delay to reply to you.
I’ll to reply to everyone and provide more detail on what we do and what we
want to achieve.
John wrote: Is there a specific reason why you have to deserialize it?
The question can be understood in two ways. The first is why do you want to use
PXD? Our Response is simple because we read that is was faster that the
classical Serialisation. The Second is why do you use serialisation? To be
honest with you, I never think about that. We use serialisation because in our
project, a Java class named “publisher” put messages to the cache (client
cache) and serialisation support is required. (The publisher uses a client
cache to access to the Geode members since we don’t want to increase the memory
size of the application using the publisher. Nevertheless, if we learn that
using a classical cache we will improve significantly the message insertion in
the cache we can do it).
We use Geode in the following way: For each process running on our product, the
publisher pushes many message type in regions (one Region per message type)
then when the process is over, a last message is put in its Region.
We capture that event through an Async Queue and an Async Event Handler gather
the messages related to the process from different regions and aggregate them.
AFAIK, we need to deserialize the object from the event if we want to use the
message in the handler.
private void aggregateEvent(AsyncEvent<String, BPELEnterpriseEvent> geodeEvent)
{
BPELEnterpriseEvent bpelEventStop = geodeEvent.getDeserializedValue();
May be using pdx feature would in that case would be more Efficient ? I don’t
know.
Otherwise, for the moment on low resource machines, we are close to our
objectives and aggregate around 10K events per second with one server and hope
processing 3 to 4 more event after optimisation and migration on decent servers.
Regards
Paul
From: Dave Barnes [mailto:[email protected]]
Sent: 28 April 2017 19:53
To: [email protected]
Subject: Re: Client cache and pdf serialization
FYI, I created a ticket for the doc bug that John pointed out:
GEODE-2843 User Guide - example should specify <client-cache>
On Fri, Apr 28, 2017 at 11:13 AM, John Blum <[email protected]
<mailto:[email protected]> > wrote:
Hi Paul-
Answers/comments inline below...
On Fri, Apr 28, 2017 at 10:02 AM, Paul Perez <[email protected]
<mailto:[email protected]> > wrote:
Hello all
We develop a kind of aggregation features for our monitoring system.
In our geode cache, we use the pdx serialisation for our java objets. They will
not be used elsewhere.
So come two questions :
First we developed event listeners to aggregate the event of our products. Once
the handler receives the asyncEvent, we have to deserialize it. Do we use pdx
features here as well?
Is there a specific reason why you have to deserialize it? You can work with
PdxInstance(s) <http://geode.apache.org/releases/latest/javadoc/index.html>
[1] directly (getting/setting fields
<http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/PdxInstance.html#createWriter-->
[2] of the object's type without deserializing). If necessary, you can
always get the deserialized, object instance
<http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/PdxInstance.html#getObject-->
[3] of the data (providing the class type is on the classpath of the server
where AEQ Listener is firing). It is even possible to query (with OQL) PDX
serialized bytes without deserializing (unlike Java serialized bytes).
However, you should be careful in that once deserialized, Geode will retain the
deserialized form of the data in the Region. This will only cause overhead
when the data needs to be re-serialized, which is anytime the data is
persisted/overflowed to disk, replicated between peers, sent across a WAN or
sent to clients.
You can learn more here
<http://geode.apache.org/docs/guide/11/developing/data_serialization/gemfire_pdx_serialization.html>
[4].
[1] http://geode.apache.org/releases/latest/javadoc/index.html
[2]
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/PdxInstance.html#createWriter--
[3]
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/pdx/PdxInstance.html#getObject--
[4]
http://geode.apache.org/docs/guide/11/developing/data_serialization/gemfire_pdx_serialization.html
Seconds question: one of our application use a client cache for memory issues.
As far as we understood, pdx serialization cannot be defined in the cache.xml.
So, if the client cache doesn't support pdx so are we forced to use the
classical java serialization to send the ojects to the servers?
Why do you think that? See here
<http://geode.apache.org/docs/guide/11/reference/topics/client-cache.html#cc-pdx>
[5].
Unfortunately, the example is WRONG, as it is using the "peer" cache element
tag (<cache> instead of <client-cache>), but this reference ([5]) is from
client-cache section and it is possible to configure PDX on the client. You
must configure PDX serialization on the client to serialize bytes to the server
from the client using PDX. Server-side PDX configuration maybe optional if you
have nothing running in the servers directly (e.g. a Function or CacheLoader)
since the server will retain the data in the form that it receives, unless, as
I mention, above, something were to cause the data to be deserialized. Then,
you need to specify your serialization strategy, e.g. either by your
application domain objects implementing
<http://geode.apache.org/docs/guide/11/developing/data_serialization/use_pdx_serializable.html>
PdxSerializable [6](invasive, and not recommended) or by implementing a
<http://geode.apache.org/docs/guide/11/developing/data_serialization/use_pdx_serializer.html>
PdxSerializer [7] (or perhaps using a OOTB impl...
ReflectionBasedAutoSerializer
<http://geode.apache.org/docs/guide/11/developing/data_serialization/auto_serialization.html>
[8], though personally, I think implementing your own PdxSerializer
implementations for your application domain objects is better... you can even
get creative using the Composite Software Design Pattern to create per domain
type serializers but treat it as 1 since Geode only allows you to register 1).
Anyway, it is possible to use PDX serialization on the client.
Hope this helps!
Cheers,
John
[5]
http://geode.apache.org/docs/guide/11/reference/topics/client-cache.html#cc-pdx
[6]
http://geode.apache.org/docs/guide/11/developing/data_serialization/use_pdx_serializable.html
[7]
http://geode.apache.org/docs/guide/11/developing/data_serialization/use_pdx_serializable.html
[8]
http://geode.apache.org/docs/guide/11/developing/data_serialization/auto_serialization.html
Thank you for your response
Paul
--
-John
john.blum10101 (skype)
