wjones127 commented on PR #12775: URL: https://github.com/apache/arrow/pull/12775#issuecomment-1151339269
> I don't know what the exact requirements for this are, but if performance is not important then do note there are already conversion utilities in stl.h that go back and forth between arrow::Table and std::vector<std::tuple<...>>. One of the intended use case is designing performant connectors for NoSQL databases, who typically provide results as a vector or iterator of row / document objects. For example, DynamoDB's C++ SDK returns `Aws::Map<Aws::String, Aws::DynamoDB::Model::AttributeValue>`. Cassandra has a similar class. More generally, there are a lot of existing systems that work off of rows that an adopter of Arrow may need to connect to. Looking that the utilities in stl.h, one thing I haven't explored is whether `arrow::stl::ConversionTraits<T>` and `arrow::CTypeTraits<T>`. > Generally I have mixed feelings about exposing and maintaining this API. On the one hand it can ease writing conversions a bit, on the other hand it doesn't really abstract the task a lot anyway (mostly it seems to translate between iterator <-> record batch sequence <-> table). I agree that the new API doesn't do that much on the task; the translation it does between iterator, RB, and tables is mostly for convenience. The main thing it does is establish a pattern of converting one batch at a time, and potentially operating on a smaller batch size than a table is currently grouped in for cases where the inner conversion loops performs better with smaller batches. It's hard to make an API that general enough that handles most of the conversion; that's why most of this PR is an example demonstrating how to use Arrow's `VisitXInline` methods, type traits, and conversions to C types to implement row conversions. Those features make writing the row-conversions a lot easier, but they aren't well-documented for end users. If we think the new API doesn't add much, I could just fold its logic into the example and this could just be a docs PR 🤷 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
