Hi everyone, I'm trying to debug a mysterious memory blowout in a Python batch job, and one of the angles I'm exploring is that this may have to do with the way it's reading data. This job is reading from bigtable, which is ultimately fetching the actual data with a unidirectional streaming "read rows" RPC. This takes a single request and returns a sequence of data chunks, the higher-level client reshapes this into an iterator over the individual data cells, and those are consumed by the higher-level program, so that the next response proto is consumed once the program is ready to parse it.
Something I can't remember about gRPC internals: What, if anything, is the pushback mechanism in unidirectional streaming? In the zero-pushback case, it would seem that a server could yield results at any speed, which would be accepted by the client and stored in gRPC's internal buffers until it got read by the client code, which could potentially cause a large memory blowout if the server wrote faster than the client read. Is this in fact the case? If so, is there any good way to instrument and detect if it's happening? (Some combination of gRPC debug flags, perhaps) If not, is there some pushback mechanism I'm not thinking of? (Alas, I can't change the protocol in this situation; the server is run by someone else) Yonatan -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAFk%3DnbTrrgEzDEDiczJgHa2v_3Q1c6pL366fV%3Dfd5wkMHW6%3D7g%40mail.gmail.com.
