Gotcha. Billie just noted in IRC that, when Accumulo has custom RPC code
a long time ago, Keith's experiment showed Thrift 500x faster. That's
probably how we got started.
Since then, I think it's just been a good solution that works well in
the end (development work can be hard). The lack of an integrated
serialization and transport system is what kept up with Thrift. I could
only come up with gRPC that could be a true replacement, but I may be
missing others.
[email protected] wrote:
I'm not suggesting that we replace Thrift (nor am I signing up to do it), just
asking for the basis of the decision and if its time to revisit. I'm totally ok
with a 'no' answer.
----- Original Message -----
From: "Josh Elser"<[email protected]>
To: [email protected]
Sent: Tuesday, December 1, 2015 2:36:18 PM
Subject: Re: State of our RPCs
To play devil's advocate: I'm not sure if it's quite that simple. For
example, Avro has been around since 2009, but I don't think it'd be fair
to consider Avro circa 2009 to Avro circa 2015.
David Medinets wrote:
What new protocols have been introduced since the Thrift decisions? Can
someone provide pros and cons for that limited set of protocols?
On Tue, Dec 1, 2015 at 1:02 PM,<[email protected]> wrote:
What was it about Thrift that drove us to use it? Was it the bindings for
multiple languages? Should this decision be revisited?
----- Original Message -----
From: "Josh Elser"<[email protected]>
To: "dev"<[email protected]>
Sent: Tuesday, December 1, 2015 12:49:26 PM
Subject: State of our RPCs
Hi --
My adventures in Thrift as a part of ACCUMULO-4065 are finally coming to
a close, it seems. The briefest summary I can give is that our hack to
work around an 0.9.0->0.9.1 compatibility issue ended up creating a bug
in a very obtuse case (when a server answering a oneway Thrift call
threw an RTE or an Error).
Given some other recent chatter in the project, I'm left wondering: what
next?
We've long considered Thrift to be a very useful tool, but extremely
scary to upgrade. I think this is just another sign of this. This leaves
me asking, how do we fix this?
Best as I understand it, Thrift is still a relatively active project (at
least their mailing list archives shows it). My impression is that the
Java library is much less-so. Most of our issues to me that they
ultimately stem from incompatibilities between libthrift versions and
uncaught performance regressions.
Assuming that to be true, do we need to make a coordinated effort to
improve the upstream libthrift code? Become a part of their community,
focusing on preventing these sorts of issues from ever filtering down to
us? Help them generate and follow compatibility guidelines?
I feel like our strategy over the past few years has been to "avert your
eyes" -- if we don't touch it, it'll hopefully be ok. Perhaps we need to
try something new. Thoughts?
- Josh
