bigluck opened a new issue, #170:
URL: https://github.com/apache/iceberg-python/issues/170
### Feature Request / Improvement
Ciao @Fokko
Seems like `table.scan()` supports a limited set of `filter` conditions, and
it fails when a user specifies a complex one.
In my case, I have this input query:
```sql
SELECT * FROM wind_energy_sensor_data where observed_at::date = '2020-03-03'
```
Once mapped into a list of fields and filters using sqlglot, I get:
```
{
"filter": "CAST(observed_at AS DATE) = '2020-03-03'",
"name": "wind_energy_sensor_data",
"projections":
[
"*",
"observed_at"
]
}
```
But then when I pass the filter to the `table.scan()` function, it raises
this `ParseException`:
```python
scan = table.scan(
^^^^^^^^^^^
File "/pip/pyiceberg/pyiceberg/table/__init__.py", line 473, in scan
return DataScan(
^^^^^^^^^
File "/pip/pyiceberg/pyiceberg/table/__init__.py", line 773, in __init__
super().__init__(table, row_filter, selected_fields, case_sensitive,
snapshot_id, options, limit)
File "/pip/pyiceberg/pyiceberg/table/__init__.py", line 629, in __init__
self.row_filter = _parse_row_filter(row_filter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/pip/pyiceberg/pyiceberg/table/__init__.py", line 603, in
_parse_row_filter
return parser.parse(expr) if isinstance(expr, str) else expr
^^^^^^^^^^^^^^^^^^
File "/pip/pyiceberg/pyiceberg/expressions/parser.py", line 270, in parse
return boolean_expression.parse_string(expr, parse_all=True)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/pip/pyparsing/pyparsing/core.py", line 1197, in parse_string
raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected expr, found '(' (at char 4),
(line:1, col:5)
```
How hard is it to extend support for functional filters (as CAST() in my
case)? And more importantly, is it something that makes sense to have, or
pyiceberg (and in general iceberg) expect these types of conditions in a
different format?
thanks,
Luca
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
