Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-06-06 Thread Galder Zamarreno
@Manik, great to hear from you! I also agree that gRPC brings a lot of benefits to our client/server architecture. In fact, I'd personally be happy to trade off "some" performance for all the benefits it brings. @Adrian I think you hit the nail with this: "...make it possible for a third party to

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-06-06 Thread Sanne Grinovero
Thanks Manik! Great to hear some feedback from you, especially as you have way more experience with gRPC. Beyond helping to develop (and maintain!) clients for a wider range of programming languages - it would also help to provide both a "traditional" and a non-blocking client for each such

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-06-06 Thread Manik Surtani
Hello everyone! (Wow, it's been a while since I dropped by and said hello... ) Super-interesting discussion. Adrian: "What is the approach you envision regarding the IDL? Should we strive for a pure IDL definition of the service? That could be an interesting approach that would make it possible

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Galder Zamarreno
On Wed, May 30, 2018 at 5:00 PM Radim Vansa wrote: > On 05/30/2018 02:53 PM, Sanne Grinovero wrote: > > On 30 May 2018 at 13:26, Adrian Nistor wrote: > >> Yest, the client needs that hash but that does not necessarily mean it > >> has to compute it itself. > >> The hash should be applied to the

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Radim Vansa
On 05/30/2018 02:53 PM, Sanne Grinovero wrote: > On 30 May 2018 at 13:26, Adrian Nistor wrote: >> Yest, the client needs that hash but that does not necessarily mean it >> has to compute it itself. >> The hash should be applied to the storage format which might be >> different from the format the

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Sanne Grinovero
On 30 May 2018 at 13:26, Adrian Nistor wrote: > Yest, the client needs that hash but that does not necessarily mean it > has to compute it itself. > The hash should be applied to the storage format which might be > different from the format the client sees. So hash computation could be > done on

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Adrian Nistor
Yest, the client needs that hash but that does not necessarily mean it has to compute it itself. The hash should be applied to the storage format which might be different from the format the client sees. So hash computation could be done on the server, just a thought. On 05/30/2018 02:47 PM,

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Emmanuel Bernard
On Wed 18-05-30 11:16, Galder Zamarreno wrote: >On Tue, May 29, 2018 at 8:57 PM Adrian Nistor wrote: > >> Vittorio, a few remarks regarding your statement "...The alternative to >> this is to develop a protostream equivalent for each supported language and >> it doesn't seem really feasible to

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Radim Vansa
On 05/30/2018 12:46 PM, Adrian Nistor wrote: > Thanks for clarifying this Galder. > Yes, the network layer is indeed the culprit and the purpose of this > experiment. > > What is the approach you envision regarding the IDL? Should we strive > for a pure IDL definition of the service? That could

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Adrian Nistor
Thanks for clarifying this Galder. Yes, the network layer is indeed the culprit and the purpose of this experiment. What is the approach you envision regarding the IDL? Should we strive for a pure IDL definition of the service? That could be an interesting approach that would make it

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Adrian Nistor
Fair point. That's why protobuf's Any has a type url inside, exactly for such flexibility : https://github.com/google/protobuf/blob/master/src/google/protobuf/any.proto#L150 Well, it's not a mime-type as per infinspan but close enough. On 05/30/2018 01:22 PM, Gustavo Fernandes wrote: On Wed,

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Gustavo Fernandes
On Wed, May 30, 2018 at 10:56 AM, Adrian Nistor wrote: > The oneof and WrappedMessage solve the same problem but in a different way. > Oneof has the nasty effect that in ties the service model to the user data > model. > The user data model is only "static" at storage level (guided by

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Adrian Nistor
The oneof and WrappedMessage solve the same problem but in a different way. Oneof has the nasty effect that in ties the service model to the user data model. Even if it seems like just one more line of code to add when a new user type is introduced, it is one line of code in the wrong place

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Galder Zamarreno
On Tue, May 29, 2018 at 8:57 PM Adrian Nistor wrote: > Vittorio, a few remarks regarding your statement "...The alternative to > this is to develop a protostream equivalent for each supported language and > it doesn't seem really feasible to me." > > No way! That's a big misunderstanding. We do

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Vittorio Rigamonti
On Tue, May 29, 2018 at 8:59 PM, Adrian Nistor wrote: > So you assume the two are separate, Emmanuel. So do I. > > But in the current PoC the user data model is directly referenced by the > service model interface (KeyMsg and ValueMsg are oneofs listing all > possible user application types???).

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Vittorio Rigamonti
On Tue, May 29, 2018 at 8:49 PM, Adrian Nistor wrote: > Vittorio, a few remarks regarding your statement "...The alternative to > this is to develop a protostream equivalent for each supported language and > it doesn't seem really feasible to me." > > No way! That's a big misunderstanding. We do

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-30 Thread Vittorio Rigamonti
Thanks Emmanuel, actually I started this thread just to describe what I did, I probably forgot an "An" at the beginning of the subject :) Vittorio On Tue, May 29, 2018 at 6:20 PM, Emmanuel Bernard wrote: > Right. Here we are talking about a gRPC representation of the client > server

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Adrian Nistor
So you assume the two are separate, Emmanuel. So do I. But in the current PoC the user data model is directly referenced by the service model interface (KeyMsg and ValueMsg are oneofs listing all possible user application types???). I was assuming this hard dependency was there just to make

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Adrian Nistor
Vittorio, a few remarks regarding your statement "...The alternative to this is to develop a protostream equivalent for each supported language and it doesn't seem really feasible to me." No way! That's a big misunderstanding. We do not need to re-implement the protostream library in C/C++/C#

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Emmanuel Bernard
Right. Here we are talking about a gRPC representation of the client server interactions. Not the data schema stored in ISPN. In that model, the API is compiled by us and handed over as a package. > On 29 May 2018, at 15:51, Sanne Grinovero wrote: > > > >> On 29 May 2018 at 13:45, Vittorio

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Sanne Grinovero
On 29 May 2018 at 13:45, Vittorio Rigamonti wrote: > Thanks Adrian, > > of course there's a marshalling work under the cover and that is reflected > into the generated code (specially the accessor methods generated from the > oneof clause). > > My opinion is that on the client side this could be

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Vittorio Rigamonti
Thanks Adrian, of course there's a marshalling work under the cover and that is reflected into the generated code (specially the accessor methods generated from the oneof clause). My opinion is that on the client side this could be accepted, as long as the API are well defined and documented:

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Bela Ban
FYI, I've also been looking into gRPC, as a tool to provide a (JGroups) version-independent service that all traffic is sent to and received from during a rolling cluster upgrade [1]. The focus of this is *version independence*, ie. have 3.6 and 4.x nodes talk to each other. A non-requirement

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-29 Thread Galder Zamarreno
Hi all, @Vittorio, thanks a lot for working on this! Let me explain some of the background behind this effort so that we're all on the same page: The biggest problem I see in our client/server architecture is the ability to quickly deliver features/APIs across multiple language clients. Both

Re: [infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-28 Thread Adrian Nistor
Hi Vittorio, thanks for exploring gRPC. It seems like a very elegant solution for exposing services. I'll have a look at your PoC soon. I feel there are some remarks that need to be made regarding gRPC. gRPC is just some nice cheesy topping on top of protobuf. Google's implementation of

[infinispan-dev] Infinispan client/server architecture based on gRPC

2018-05-28 Thread Vittorio Rigamonti
Hi Infinispan developers, I'm working on a solution for developers who need to access Infinispan services through different programming languages. The focus is not on developing a full featured client, but rather discover the value and the limits of this approach. - is it possible to