Weston Pace created ARROW-17521: ----------------------------------- Summary: [Python] Add python bindings for NamedTableProvider for Substrait consumer Key: ARROW-17521 URL: https://issues.apache.org/jira/browse/ARROW-17521 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Weston Pace
The C++ Substrait consumer currently supports a named table provider to handle the NamedTable relation: {noformat} using NamedTableProvider = std::function<Result<compute::Declaration>(const std::vector<std::string>&)>; static NamedTableProvider kDefaultNamedTableProvider; /// Options that control the conversion between Substrait and Acero representations of a /// plan. struct ConversionOptions { /// \brief How strictly the converter should adhere to the structure of the input. ConversionStrictness strictness = ConversionStrictness::BEST_EFFORT; /// \brief A custom strategy to be used for providing named tables /// /// The default behavior will return an invalid status if the plan has any /// named table relations. NamedTableProvider named_table_provider = kDefaultNamedTableProvider; }; {noformat} This is very useful for testing and experimenting as it allows you to provide tables from memory (using a table_source node for example). We should add pyarrow bindings. I don't think they need to expose the full compute::DeclarationInfo range of table sources. A simple approach might be a function that, given a list of names, returns either a table, an iterable of batches, or a record batch reader. -- This message was sent by Atlassian Jira (v8.20.10#820010)