Re: [PR] Implement `tree` explain for `ArrowFileSink` [datafusion]
alamb commented on code in PR #15206: URL: https://github.com/apache/datafusion/pull/15206#discussion_r1996139918 ## datafusion/sqllogictest/test_files/explain_tree.slt: ## @@ -1716,14 +1716,21 @@ TO 'test_files/scratch/explain_tree/1.json'; physical_plan 01)┌───┐ 02)│DataSinkExec │ -03)└─┬─┘ -04)┌─┴─┐ -05)│ DataSourceExec │ -06)│ │ -07)│bytes: 2672│ -08)│ format: memory │ -09)│ rows: 1 │ -10)└───┘ +03)│ │ Review Comment: 😍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Implement `tree` explain for `ArrowFileSink` [datafusion]
irenjj commented on code in PR #15206: URL: https://github.com/apache/datafusion/pull/15206#discussion_r1995358596 ## datafusion/sqllogictest/test_files/explain_tree.slt: ## @@ -1711,35 +1711,58 @@ physical_plan query TT explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) -TO 'test_files/scratch/explain_tree/1.json'; +TO '/tmp/1.json'; Review Comment: Thanks @alamb ,That's a good idea! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Implement `tree` explain for `ArrowFileSink` [datafusion]
alamb commented on PR #15206: URL: https://github.com/apache/datafusion/pull/15206#issuecomment-2721935376 This is so close thank you @irenjj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Implement `tree` explain for `ArrowFileSink` [datafusion]
alamb commented on code in PR #15206:
URL: https://github.com/apache/datafusion/pull/15206#discussion_r1993919740
##
datafusion/sqllogictest/test_files/explain_tree.slt:
##
@@ -1711,35 +1711,58 @@ physical_plan
query TT
explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'),
(3, 'baz', 3, '2023-01-03'))
-TO 'test_files/scratch/explain_tree/1.json';
+TO '/tmp/1.json';
Review Comment:
Maybe we can save the original path for display - perhaps we could save the
`output_url` here as a field on DataSinkConfig 🤔
https://github.com/apache/datafusion/blob/db45ff3eea33c0e3ad607ce1abff266a9956ab22/datafusion/core/src/physical_planner.rs#L500-L499
```rust
FileSinkConfig {
...
/// The unresolved URL specified by the user
original_url: String
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] Implement `tree` explain for `ArrowFileSink` [datafusion]
alamb commented on code in PR #15206: URL: https://github.com/apache/datafusion/pull/15206#discussion_r1993896191 ## datafusion/sqllogictest/test_files/explain_tree.slt: ## @@ -1711,35 +1711,58 @@ physical_plan query TT explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) -TO 'test_files/scratch/explain_tree/1.json'; +TO '/tmp/1.json'; physical_plan 01)┌───┐ 02)│DataSinkExec │ -03)└─┬─┘ -04)┌─┴─┐ -05)│ DataSourceExec │ -06)│ │ -07)│bytes: 2672│ -08)│ format: memory │ -09)│ rows: 1 │ -10)└───┘ +03)│ │ +04)│ file:///tmp/1.json│ Review Comment: Can we also possibly add `format: csv` ? ## datafusion/sqllogictest/test_files/explain_tree.slt: ## @@ -1711,35 +1711,58 @@ physical_plan query TT explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) -TO 'test_files/scratch/explain_tree/1.json'; +TO '/tmp/1.json'; Review Comment: Yeah, we need to avoid printing the entire path. Let me see if I can find some way -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Implement `tree` explain for `ArrowFileSink` [datafusion]
irenjj commented on code in PR #15206: URL: https://github.com/apache/datafusion/pull/15206#discussion_r1993622566 ## datafusion/sqllogictest/test_files/explain_tree.slt: ## @@ -1711,35 +1711,58 @@ physical_plan query TT explain COPY (VALUES (1, 'foo', 1, '2023-01-01'), (2, 'bar', 2, '2023-01-02'), (3, 'baz', 3, '2023-01-03')) -TO 'test_files/scratch/explain_tree/1.json'; +TO '/tmp/1.json'; Review Comment: It will also print my local directory if I use a relative path.👀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
