Paddy Horan created ARROW-8197: ---------------------------------- Summary: [Rust] DataFusion "create_physical_plan" returns incorrect schema? Key: ARROW-8197 URL: https://issues.apache.org/jira/browse/ARROW-8197 Project: Apache Arrow Issue Type: Bug Components: Rust, Rust - DataFusion Affects Versions: 0.15.1 Reporter: Paddy Horan
I am using DataFusion in a situation where I know there will only be a single file. DataFusion currently collects all batches into a vector. As I am writing the data back out I want to work with an iterator instead of a vector. I have something as follows: {code:java} let plan = ctx.create_logical_plan(&sql).unwrap(); let plan = ctx.optimize(&plan).unwrap(); dbg!(plan.schema()); // Returns field names let plan = ctx.create_physical_plan(&plan, batch_size).unwrap(); dbg!(plan.schema()); // Returns c0, c1, etc{code} Maybe this is expected after turning the plan into a physical plan? I can change the schema of the returned batches, would this be the recommended way to address this or is their something in DataFusion I should leverage to do this? -- This message was sent by Atlassian Jira (v8.3.4#803005)