Absolutely its a implementation detail.
JSON: Surely we can consider ValueHeader. But then every client(and message)
needs to send that. Using metadata its a optional.
From: Dan Smith <dsm...@pivotal.io>
To: Udo Kohlmeyer <ukohlme...@pivotal.io>
Cc: dev@geode.apache.org
Sent: Tuesday, May 2, 2017 11:39 AM
Subject: Re: [gemfire-dev] New Client-Server Protocol Proposal
> IsPartialMessage: This flag gives us ability to send partial message
> without serializing the whole key-value(request). lets say I execute
> function at server and function just returns "arraylist of object". And the
> serialized size of ""arraylist of object"" is quite big( > 2gb).
>
My point about these fields is that it really seems like this stuff should
be handled by different layers. Ideally you would have a fragmentation
layer that is invisible to people writing specific messages, so that
messages are automatically fragmented if they get to large. Think about how
a TCP socket works - you just write data and it is automatically
fragmented. Or are you expecting each individual message type to have it's
own way to doing fragmentation, but it should set this header down in your
protocol layer? That seems really messy.
JSON: this is a feature we want to introduce, where client can send JSON
> string and we want to save that JSON string into pdx.
Same thing here, JSON support sounds great, but having a header field of
JSON_KEY seems like a hacky way to do that. It seems like that might belong
in your ValueHeader.
On Tue, May 2, 2017 at 10:20 AM, Udo Kohlmeyer <ukohlme...@pivotal.io>
wrote:
> Hey Dan,
>
> Imo, having a standardized, versioned definition for GET, PUT, PUTALL,
> etc. message, that is encoded/decoded in a manner that multiple clients
> (written in many other languages) can encode/decode these messages, is
> paramount.
>
> Having the standardized operational messages(GET,PUT,etc.) transported
> using the function service vs a more direct operation handler, that is
> another discussion and is something that should be investigated.
>
> My immediate concerns regarding "normal" operations over the function
> service are:
>
> 1. I don't believe the current function service is "stream" enabled,
> and would require some potential rework for subscription-based operations
> 2. Can the function service handle the extra load?
> 3. Is the function service "lean" enough to sustain acceptable
> throughput? The current client/server protocol averages around
> 40,000-50,000 messages/second.
> 4. There are some messages that are passed between the client <->
> locator. Given that the function service is "server" specific, this
> approach would not work for locators, where a different transport mechanism
> is required. (but this is not a show stopper if function service proves to
> be viable)
> 5. How much effort would be required to make the "old" function
> service, handle the new messages, ensuring that the current behavior is
> preserved.
>
> As per a previous discussion we had, I believe that the "function-like"
> behavior (retry, HA, write vs read optimized) can incorporated into the
> processing layer on the server. In that way all messages can benefit from
> that behavior. In addition to this, if we have a single mechanism that will
> handle messages, retry, HA, read/write optimizations, is preferable to
> having a few "bespoke" implementations. So either approach (new message
> handling) vs function service, will be preferable.
>
> "*The advantage of this approach is that if someone just builds a driver
> that only supports function execution and whatever serialization framework
> is required to serialize function arguments, they already have an API that
> application developers could use to do pretty much anything they wanted to
> do on the server. Having a Region object with methods like get and put on
> it could just be a little syntatic sugar on top of that.*"
>
> It can be argued that having a standard client/server message, with
> standardized encoding/decoding, is the same as using function execution.
> Both require a little syntactic sugar to add new functionality to an
> already standardized message.
>
> --Udo
> On 5/1/17 17:27, Dan Smith wrote:
>
> I think any new client driver or server we develop might want to
> incorporate function execution at lower level than region operations like
> get and put, etc. We could then easily build operations like GET, PUT,
> PUTALL, etc. on top of that by making them functions. The original client
> protocol isn't designed like that because it pre-dates function execution.
>
> The current function execution API is a little clunky and needs some work.
> But what it does do is provide the fundamental logic to target operations
> at members that host certain keys and retry in the case of failure.
>
> The advantage of this approach is that if someone just builds a driver
> that only supports function execution and whatever serialization framework
> is required to serialize function arguments, they already have an API that
> application developers could use to do pretty much anything they wanted to
> do on the server. Having a Region object with methods like get and put on
> it could just be a little syntatic sugar on top of that.
>
> -Dan
>
> On Fri, Apr 28, 2017 at 2:49 PM, Udo Kohlmeyer <ukohlme...@pivotal.io>
> wrote:
>
>> Hi there Geode community,
>>
>> The new Client-Server protocol proposal is available for review.
>>
>> It can be viewed and commented on https://cwiki.apache.org/confl
>> uence/display/GEODE/New+Client+Server+Protocol
>>
>> --Udo
>>
>>
>
>
