Hi there Nan,

Given that you would have 2 separate regions, you should not ever run into the situation where properties and data are significantly "out-of-sync". If you were to use the same WAN Sender for both regions, the data and properties should be sent in the same batch or subsequent batches.

This approach is not "atomic" and yes, you could end up in a situation where there might be small point in time where they are "out-of-sync".

IF there is an absolute requirement for atomicity AND smaller region.get() operations, there are other approaches that you could use. Maybe insert the big object (properties + data) into an "ingestion" region. Then have a CacheListener split the big object into data and properties regions. WAN Replication can then be setup on the "ingestion" region, thus data + properties move atomically.

Of course, the remote cluster will have to then take the WAN replicated "big Object" and split it into the querying regions.

Once again, many more moving parts.... and possibly overkill if you don't have the requirement of atomic and eventual consistency (within 1 or batches) is good enough.

--Udo


On 8/30/17 10:04, Xu, Nan wrote:

collocating is a good suggestion, so assume I have data region and property region collocated. We need the data distributed to multiple datacenter and to save latency and bandwidth, we are going to use conflation. What I am afraid is data get conflated and old data not send out and properties get send out. So they are not exactly 1:1 match then. Suggestion?  Would be eventual consistent?(I think so)

Thanks,

Nan

*From:*Udo Kohlmeyer [mailto:[email protected]]
*Sent:* Wednesday, August 30, 2017 11:01 AM
*To:* [email protected]
*Subject:* Re: Get part of the object

Hi the Nan,

From experience I can tell you it is better to safe the byte[] to a separate (colocated region). You can then have a data and properties region, which use the same key to store the data in both regions.

The only thing you would have to manage yourself would be the CRUD operations for each. i.e If you insert properties you'll have to insert the data into the data region. If you remove/update properties you'll potentially have to do the same to the data region. Geode does not provide the mechanism to manage data splits.

--Udo

On 8/30/17 08:20, Lyndon Adams wrote:

    Yep thanks Luke.

        On 30 Aug 2017, at 16:12, Xu, Nan <[email protected]
        <mailto:[email protected]>> wrote:

        “Use selective gettter methods on the wrapper object”

        How?  Region.get() already grab the full object from network.
        Or something I am missing?

        *From:*Lyndon Adams [mailto:[email protected]]
        *Sent:*Wednesday, August 30, 2017 10:05 AM
        *To:*[email protected] <mailto:[email protected]>
        *Subject:*Re: Get part of the object

        Use selective gettter methods on the wrapper object to achieve
        this requirement.

            On 30 Aug 2017, at 16:03, Xu, Nan <[email protected]
            <mailto:[email protected]>> wrote:

            Yes, but because data is big, >100K, and other is small,
            to save the bandwidth, I don’t want to get the full object
            back but only the other part.

            *From:*Dharam Thacker [mailto:[email protected]]
            *Sent:*Wednesday, August 30, 2017 10:01 AM
            *To:*[email protected] <mailto:[email protected]>
            *Subject:*Re: Get part of the object

            Region.get() is going to return you domain object.

            So let's say yoy have domain class with

            {

            byte[] data

            Map<String,Object> other

             }

            You can say,

            Region.get(data).getOther()

            Thanks,

            Dharam

            On Aug 30, 2017 8:18 PM, "Lyndon Adams"
            <[email protected] <mailto:[email protected]>>
            wrote:

                Yes

                    On 30 Aug 2017, at 15:47, Xu, Nan <[email protected]
                    <mailto:[email protected]>> wrote:

                    Not sure I get it. you mean save a list, and list
                    comprise of 2 objects -- data and properties?

                    *From:*Lyndon Adams [mailto:[email protected]]
                    *Sent:*Wednesday, August 30, 2017 9:45 AM
                    *To:*[email protected]
                    <mailto:[email protected]>
                    *Subject:*Re: Get part of the object

                    Make the value a collection wrapper.

                        On 30 Aug 2017, at 15:39, Xu, Nan
                        <[email protected] <mailto:[email protected]>> wrote:

                        Thanks, wondering if geode can save 2 objects
                        for a single key? So I can put data and other
                        property separately. Or every key is for 1
                        object? Never use the PDX and server side
                        function, is that mean those need to be
                        deployed to the server? or there is a way to
                        programming register the function to the server?

                        Thanks,

                        Nan

                        *From:*Luke Shannon [mailto:[email protected]]
                        *Sent:*Tuesday, August 29, 2017 8:02 PM
                        *To:*[email protected]
                        <mailto:[email protected]>
                        *Subject:*Re: Get part of the object

                        +1 on Mike's suggestion for the
                        ReflectionBasedAutoSerializer

                        To get name and author you could use OQL:

                        
https://gemfire.docs.pivotal.io/geode/getting_started/querying_quick_reference.html#reference_D5CE64F5FD6F4A808AEFB748C867189E
                        
<https://urldefense.proofpoint.com/v2/url?u=https-3A__gemfire.docs.pivotal.io_geode_getting-5Fstarted_querying-5Fquick-5Freference.html-23reference-5FD5CE64F5FD6F4A808AEFB748C867189E&d=DwMFaQ&c=SFszdw3oxIkTvaP4xmzq_apLU3uL-3SxdAPNkldf__Q&r=HB5LZowSGF4DiMmOUsCX6Q&m=8beRJvUxxhwMi53HGv0iYc9bpWOpIjRLWs6L_12UPVQ&s=XWQnjvDN7lTRBBHYWEMKtnyPfsLxW_Pq0wL1ZSMK1No&e=>

                        On Tue, Aug 29, 2017 at 5:45 PM, Michael Stolz
                        <[email protected] <mailto:[email protected]>>
                        wrote:

                            You should probably use the PDX
                            ReflectionBasedAutoSerializer.

                            Then you can write a server-side function
                            that you call to get just the fields you
                            want and return them.


                            --

                            Mike Stolz

                            Principal Engineer, GemFire Product Manager

                            Mobile:+1-631-835-4771
                            <tel:%28631%29%20835-4771>

                            On Tue, Aug 29, 2017 at 4:49 PM, Xu, Nan
                            <[email protected] <mailto:[email protected]>> wrote:

                                Hi,

                                I have an object comprise of

                                MyObject{

                                byte[]  data;

                                String name;

                                 String author;

                                ….

                                Other properties;

                                }

                                Or I can use

                                MyObject{

                                Byte[] data;

                                Other allOther;

                                }

                                When I get by key, I only want to get
                                all the properties except the data
                                itself, because data could be large,
                                is that has to be done with query? Not
                                sure how the performance of query
                                compare to  region.get(key).

                                Another question is to serialize the
                                data. Do I need to upload the jar
                                containing MyObject to geode server?

                                Thanks,

                                Nan

                                
------------------------------------------------------------------------

                                This message, and any attachments, is
                                for the intended recipient(s) only,
                                may contain information that is
                                privileged, confidential and/or
                                proprietary and subject to important
                                terms and conditions available
                                athttp://www.bankofamerica.com/emaildisclaimer.
                                If you are not the intended recipient,
                                please delete this message.



                        --

                        Luke Shannon | Platform Engineering | Pivotal

                        
-------------------------------------------------------------------------


                        Mobile:416-571-9495 <tel:%28416%29%20571-9495>

                        Join the Toronto Pivotal Usergroup:
                        http://www.meetup.com/Toronto-Pivotal-User-Group/
                        
<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.meetup.com_Toronto-2DPivotal-2DUser-2DGroup_&d=DwMFaQ&c=SFszdw3oxIkTvaP4xmzq_apLU3uL-3SxdAPNkldf__Q&r=HB5LZowSGF4DiMmOUsCX6Q&m=8beRJvUxxhwMi53HGv0iYc9bpWOpIjRLWs6L_12UPVQ&s=6_3MUQQcXn4OGBkgQjyTWec_NaJU4gvp3tHRQfoV-38&e=>

                        
------------------------------------------------------------------------

                        This message, and any attachments, is for the
                        intended recipient(s) only, may contain
                        information that is privileged, confidential
                        and/or proprietary and subject to important
                        terms and conditions available
                        athttp://www.bankofamerica.com/emaildisclaimer.
                        If you are not the intended recipient, please
                        delete this message.

                    
------------------------------------------------------------------------

                    This message, and any attachments, is for the
                    intended recipient(s) only, may contain
                    information that is privileged, confidential
                    and/or proprietary and subject to important terms
                    and conditions available
                    athttp://www.bankofamerica.com/emaildisclaimer. If
                    you are not the intended recipient, please delete
                    this message.

            
------------------------------------------------------------------------

            This message, and any attachments, is for the intended
            recipient(s) only, may contain information that is
            privileged, confidential and/or proprietary and subject to
            important terms and conditions available
            athttp://www.bankofamerica.com/emaildisclaimer. If you are
            not the intended recipient, please delete this message.

        ------------------------------------------------------------------------

        This message, and any attachments, is for the intended
        recipient(s) only, may contain information that is privileged,
        confidential and/or proprietary and subject to important terms
        and conditions available
        athttp://www.bankofamerica.com/emaildisclaimer. If you are not
        the intended recipient, please delete this message.

------------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

Reply via email to