lidavidm commented on issue #3134: URL: https://github.com/apache/arrow-adbc/issues/3134#issuecomment-3062144737
As for decimal precision: there is no real solution, unless the client tries to cast everything to a common type. It may not always be able to determine a common type either, except perhaps by trying to get the schema from every worker first (and even then there may not be a common type possible depending on the precision/scale requirements), which would potentially tie up a lot of server resources and require some consideration of backpressure management. If you are talking about a Flight SQL client having to potentially dynamically unify _any_ parameterized/complex type then it gets more complicated, if we have to unify structs, timestamp units, and so on. (And if we're unifying structs, does that mean we synthesize null columns on the fly, if there's different field names for say list children, which field name gets picked, etc.) It may still be possible as an option (most of this functionality should be available in arrow-go already) but I'd like to see what @zeroshade thinks (should we move this to the mailing list to put in front of the other Flight SQL users?). Again, Dremio designed Flight SQL so I'm surprised they (you?) are running into these problems...I'm curious why Dremio apparently doesn't have a consistent schema for a single query/why the Dremio workers can't perform the required casting. I always saw Flight SQL as moving towards a relatively thin client (with most of the "smarts" focusing on retries etc.), and this pushes it back towards requiring a thick client (and at the very least, requiring an Arrow implementation capable of various casts and data transforms and not just a memory format implementation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
