Hello,
I have a system with:
Flight Client <-> Proxy <-> Flight Server
I am using BackPressure to prevent the Proxy from buffering the entire
stream.
For the Proxy, when *getStream()* is called, it will handle it in another
thread. In the other thread, it will call *getStream()* on the Flight
Server and then run through a loop like this:
while (serverStream.next()) {
// Prepare client stream
while (true) {
result = backPressure.waitForListener(timeout);
if (result == READY) {
// Copy to client stream
} else if (result == TIMEOUT) {
// cancel server stream
// notify client of error
return;
}
}
}
I have seen some issues when the stream is large. If the client is fast,
then it is able to receive the complete stream. If the client is slower,
then it will usually fail. One of the failures that I have looked into has
the *backPressure.waitForListener(timeout)* call in the Flight Server
timeout. The timeout was reasonably large. During this time the Flight
Client was continuing to read batches. The Proxy was continuing to loop
through the outer while loop, but the Flight Server never saw the Proxy as
ready.
Why wasn't the Flight Server able to see the Proxy as ready when the
timeout happened?
