ftschafer commented on issue #1325: URL: https://github.com/apache/arrow-go/issues/1325#issuecomment-5878340460
Thanks for the quick response. Regarding the questions: 1. Strictly record batches, never dictionaries. Our producer is a single ipc.Writer per stream built with only WithAllocator + WithSchema, no WithDictionaryDeltas and no dictionary-encoded columns. The first payload carries the schema message and every payload after it is record batches only, so your ContinueFrom precondition matches our shape. 2. Always in order, single logical stream, never interleaved. Each decoder is bound 1:1 to one gRPC Write stream and fed payloads sequentially in the order the client sent them. The synchronous ContinueFrom(src io.Reader) signature is sufficient for us. So yes, ContinueFrom(io.Reader) would work for us, but with one request: please make ContinueFrom fail loudly if a schema message unexpectedly appears on the continuation reader, rather than returning empty. A clear error is far easier to catch than dropped rows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
