Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-10 Thread Josh Elser
My hope is that we could fix this "once" by adding another component :) To Francis' point, a standardized solution around this problem in which we use some other tech (e.g. haproxy) would be directly reusable for an Avatica client in any language. I think a very nice contribution would be tak

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-10 Thread JD Zheng
Totally make sense. The only inconvenience is that every user has to repeat the same work by setting up the sticky sessions. -Jiandan > On Aug 9, 2018, at 1:08 PM, Josh Elser wrote: > > The decision of avoiding "routing logic" in the client was that other systems > can do a better job than w

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-10 Thread JD Zheng
Somehow, my impression was that the Avatica implementation was such that the states are carried in the request/response so that server does not need to maintain any specific state for a jdbc session. Just as Julian described. Obviously I was wrong. Just curious what is the exact states maintain

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-09 Thread Francis Chuang
This was also an issue I considered for the Avatica Go client when working on the last release. I also came to the conclusion that while implementation is not insurmountable, the amount of effort required is not insignificant. If you're able to push this out to a service mesh such as Istio or

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-09 Thread Josh Elser
The decision of avoiding "routing logic" in the client was that other systems can do a better job than we can in Avatica. There are other systems which are specifically designed for doing this -- it's a clear architectural boundary that says one Avatica client expects to talk to one Avatica ser

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-09 Thread Julian Hyde
One important question is: what state is shared among servers? Some pieces of state: connection state, statement state, result set state (parameter values and position in a scroll). It is not unusual for a cluster of servers to use a shared cache for some of the larger / slower-changing pieces o

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-09 Thread JD Zheng
Hi, Josh, Thank you for sharing your experience and the nice writeup in the hortonworks website too. It’s very helpful. I am just curious why “not implement routing logic in the client” was one of the original design goals? Doesn’t it make easier to use Avatica? I agree that the sharing state b

Re: Avatica client can not talk to Multiple Avatica Servers issue

2018-08-09 Thread Josh Elser
Hi Jiandan, Glad you found my write-up on this. One of the original design goals was to *not* implement routing logic in the client. Sticky-sessions is by far the easiest way to implement this. There is some retry logic in the Avatica client to resubmit requests when a server responds that i

Avatica client can not talk to Multiple Avatica Servers issue

2018-08-08 Thread JD Zheng
Hi, Our query engine is using calcite as parser/optimizer and enumerable as runtime if needed to federate different storage engines. We are trying to enable JDBC access to our query engine. Everything works smoothly when we only have one calcite/avatica server. However, JDBC calls will fail i