Re: [PR] Simplify expressions passed to table functions [datafusion]

2025-06-14 Thread via GitHub 


comphead merged PR #16388:
URL: https://github.com/apache/datafusion/pull/16388


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] Simplify expressions passed to table functions [datafusion]

2025-06-13 Thread via GitHub 


alamb commented on code in PR #16388:
URL: https://github.com/apache/datafusion/pull/16388#discussion_r2145600999


##
datafusion/core/src/execution/session_state.rs:
##
@@ -1675,6 +1675,13 @@ impl ContextProvider for SessionContextProvider<'_> {
 .get(name)
 .cloned()
 .ok_or_else(|| plan_datafusion_err!("table function '{name}' not 
found"))?;
+let dummy_schema = DFSchema::empty();

Review Comment:
   I agree long term it would be great to have table functions support 
referencing other relations.
   
   However, we would need to update the API more than just this -- right now 
TableFunctions have no way of getting inputs (they return TableProviders)
   
   We would have to work out what the API would look like if they got an input 
(perhaps they get an input ExecutonPlan) 
   
   As you mention, we would also probably have to figure out planning for 
LATERALs, etc



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] Simplify expressions passed to table functions [datafusion]

2025-06-13 Thread via GitHub 


jonathanc-n commented on code in PR #16388:
URL: https://github.com/apache/datafusion/pull/16388#discussion_r2144347173


##
datafusion/core/src/execution/session_state.rs:
##
@@ -1675,6 +1675,13 @@ impl ContextProvider for SessionContextProvider<'_> {
 .get(name)
 .cloned()
 .ok_or_else(|| plan_datafusion_err!("table function '{name}' not 
found"))?;
+let dummy_schema = DFSchema::empty();

Review Comment:
   I think we eventually want to reference existing columns
   
   For example:
   ``` SELECT * FROM t, range(t.id, t.id + 5); ```
   
   I think the solution to this instead would be to have the a `ExprSimplifier` 
optimizer rule where it can reference the column values from the `LATERAL` 
statement. @Lordworms WDYT?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] Simplify expressions passed to table functions [datafusion]

2025-06-13 Thread via GitHub 


jonathanc-n commented on code in PR #16388:
URL: https://github.com/apache/datafusion/pull/16388#discussion_r2144347173


##
datafusion/core/src/execution/session_state.rs:
##
@@ -1675,6 +1675,13 @@ impl ContextProvider for SessionContextProvider<'_> {
 .get(name)
 .cloned()
 .ok_or_else(|| plan_datafusion_err!("table function '{name}' not 
found"))?;
+let dummy_schema = DFSchema::empty();

Review Comment:
   I think we eventually want to reference existing columns
   
   For example:
   ``` SELECT * FROM t, range(t.id, t.id + 5); ```
   
   I think the solution to this instead would be to have the a `ExprSimplifier` 
optimizer rule where it can reference the column values. @Lordworms WDYT?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] Simplify expressions passed to table functions [datafusion]

2025-06-13 Thread via GitHub 


jonathanc-n commented on code in PR #16388:
URL: https://github.com/apache/datafusion/pull/16388#discussion_r2144347173


##
datafusion/core/src/execution/session_state.rs:
##
@@ -1675,6 +1675,13 @@ impl ContextProvider for SessionContextProvider<'_> {
 .get(name)
 .cloned()
 .ok_or_else(|| plan_datafusion_err!("table function '{name}' not 
found"))?;
+let dummy_schema = DFSchema::empty();

Review Comment:
   I think the original issue mentions being able to reference existing columns.
   
   For example:
   ```sql SELECT * FROM t, range(t.id, t.id + 5); ```
   
   I think the solution to this instead would be to have the a `ExprSimplifier` 
optimizer rule where it can reference the column values. @Lordworms WDYT?
   
   



##
datafusion/core/src/execution/session_state.rs:
##
@@ -1675,6 +1675,13 @@ impl ContextProvider for SessionContextProvider<'_> {
 .get(name)
 .cloned()
 .ok_or_else(|| plan_datafusion_err!("table function '{name}' not 
found"))?;
+let dummy_schema = DFSchema::empty();

Review Comment:
   I think the original issue mentions being able to reference existing columns.
   
   For example:
   ``` SELECT * FROM t, range(t.id, t.id + 5); ```
   
   I think the solution to this instead would be to have the a `ExprSimplifier` 
optimizer rule where it can reference the column values. @Lordworms WDYT?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] Simplify expressions passed to table functions [datafusion]

2025-06-13 Thread via GitHub 


jonathanc-n commented on code in PR #16388:
URL: https://github.com/apache/datafusion/pull/16388#discussion_r2144347173


##
datafusion/core/src/execution/session_state.rs:
##
@@ -1675,6 +1675,13 @@ impl ContextProvider for SessionContextProvider<'_> {
 .get(name)
 .cloned()
 .ok_or_else(|| plan_datafusion_err!("table function '{name}' not 
found"))?;
+let dummy_schema = DFSchema::empty();

Review Comment:
   I think the original issue mentions being able to reference existing columns.
   
   For example:
```sql SELECT * FROM t, range(t.id, t.id + 5); ``` 
   
   I think the solution to this instead would be to have the a `ExprSimplifier` 
optimizer rule where it can reference the column values. @Lordworms WDYT?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]