It was probably lost in the wall of text, but the point I was trying to
convey was that fetching all rows via protobuf crashed PQS, while fetching
it via JSON didn't. Basically, the only way I can get protobuf
serialization to work is to 1. bump up the PQS memory and/or 2. fetch in
batches, while
Manoj,
re: #2 Please start by reading:
https://plumbr.io/outofmemoryerror/gc-overhead-limit-exceeded
This should give you a thorough explanation of what this error means.
You need to increase the memory footprint and/or JVM GC properties for
PQS to address your issue. When you request all
Thanks for the pointers Josh.
Here’s a set of instructions to demonstrate the performance issues I’m
seeing when using protobuf serialization. Hopefully these are comprehensive
enough to reproduce the issue. If not, please let me know and I’m happy to
provide clarifications.
Basic outline of the
On 11/5/18 10:10 PM, Manoj Ganesan wrote:
Thanks for the pointers Josh. I'm working on getting a representative
concise test to demonstrate the issue.
Meanwhile, I had one question regarding the following:
You are right that the operations in PQS should be exactly the same,
Thanks for the pointers Josh. I'm working on getting a representative
concise test to demonstrate the issue.
Meanwhile, I had one question regarding the following:
You are right that the operations in PQS should be exactly the same,
> regardless of the client you're using -- that is how this
Is the OOME issue regardless of using the Java client (sqlline-thin) and
the Python client? I would like to know more about this one. If you can
share something that reproduces the problem for you, I'd like to look
into it. The only suggestion I have at this point in time is to make
sure you
Thanks Josh for the response!
I would definitely like to use protobuf serialization, but I'm observing
performance issues trying to run queries with a large number of results.
One problem is that I observe PQS runs out of memory, when its trying to
(what looks like to me) serialize the results in
I would strongly suggest you do not use the JSON serialization.
The JSON support is implemented via Jackson which has no means to make
backwards compatibility "easy". On the contrast, protobuf makes this
extremely easy and we have multiple examples over the past years where
we've been able to
